Sovereign AI in Poland: Language Adaptation, Local Control & Cost Advantages with Marek Kozlowski
Summary
Poland’s sovereign-AI thesis is specialization, not a race against US or Chinese frontier labs. Marek Kozlowski wants Polish- or domain-adapted models that are “orders of magnitude smaller” yet match models 10 times larger inside a defined language, culture, or workflow. The payoff is local control, lower inference and deployment cost, on-premise operation, and retained technical capability even if foreign models become unavailable or legally unusable.
Frontier-model support for smaller languages may deteriorate as vendors optimize for core workloads. Kozlowski says some Claude and GPT releases have been flat or worse on Polish linguistic and cultural evaluations as developers emphasize coding and other priority markets. For enterprises building deep integrations, that creates “a huge risk”: a provider can change its target objective, degrading Polish performance and forcing a rollback or vendor migration.
Data scarcity and EU regulation define Poland’s structural disadvantage more than parameter count does. Kozlowski estimates that 90% or more of frontier training data is English or Chinese, while Polish can represent about 1% or less; PLLuM retained roughly 200 billion Polish tokens after deduplication and filtering, versus at least 1 trillion tokens he says an 8-billion-parameter model needs for stable training from random weights. European rules can make the gap worse because compliance constraints “can eliminate 80% of the data” from a prospective training corpus.
PLLuM treats human-created post-training data as its differentiating asset. Its internal tools support dozens or hundreds of annotators producing and editing instructions and preferences, while synthetic material is human-checked because “linguistically poor” instructions can degrade generation quality. The project has also published an almost 100-page recipe, samples on Hugging Face, and the principle that “open source is not only about open weights.”
The strongest near-term economics are in narrow, on-premise models serving 10 or 20 workflows—not general assistants serving thousands. Kozlowski says roughly 1,000 task-specific instructions can be enough, with more preferable, to fine-tune a smaller model to equal or sometimes exceed a giant cloud model used zero- or few-shot. Once a buyer prices 16 GPUs, energy, privacy, and operational control, “you always go through the downscaling.”
PLLuM is public infrastructure rather than a conventional venture-scale market-share play. Funded by Poland’s Ministry of Digital Affairs through a consortium that grew from six to eight institutes and universities, it prioritizes legal compliance, transparency, security, local deployment, and assistants for citizens and municipal offices over customer counts or immediate ROI. Sovereignty here means retaining “the competency and possibility” to build, even if Poland’s model is somewhat worse than the global leader.
Enterprise domain adaptation has a high data threshold that sharply limits the addressable customer base. PLLuM demonstrated continued pre-training for PKO, described as Central and Eastern Europe’s largest bank, but Kozlowski estimates a useful adaptation needs about 10 billion cleaned tokens—perhaps 30–40 billion before filtering. His blunt conclusion, “It’s not so easy to get 10 billion tokens,” makes data inventory, permissions, and curation prerequisites rather than implementation details.
The strategy ultimately rests on a disputed view of frontier progress and its cost curve. Kozlowski sees GPT-5 versus GPT-4 as evidence that improvement is becoming “horizontal,” while Nathan Labenz pushes back that exponentially larger runs—say $10 billion versus $1 billion—might still deliver major jumps. Kozlowski’s answer is demand-led: define the business task and benchmark first, because most deployments he sees do not require frontier reasoning at all.
Deep dive
1. Localization narrows the contest to ground Poland can defend
Kozlowski defines localization broadly: a model can be adapted to a national language or to a business domain. In either case, the objective is better comprehension and generation inside that boundary—not universal leadership across every benchmark and task.
The aspiration is deliberately asymmetric: build models “orders of magnitude smaller” that perform like models 10 times larger on Polish language, culture, or a selected domain. They should also be open, transparent, secure, and “as much organic as we can.”
Sovereignty is partly an option on future disruption. Foreign weights might close, licenses might change, or European use might be prohibited; Poland would then prefer a slightly weaker domestic model over having no ability to build one. “Sometimes it means more than you think.”
2. Cross-language transfer communicates, but it does not sound native
Kozlowski’s estimate is that English and Chinese contribute 90% or more of frontier-model training data, leaving all other languages under 10%; Polish can be about 1% or less. Most skills and competencies are consequently learned through English- and Chinese-language examples.
Large models compensate through transfer: a Spanish mathematics request can effectively be mapped into an English-learned solution process. Kozlowski likens this to children or other language learners mapping a new language through their native one before producing an answer.
That mechanism creates intelligible Polish without native Polish usage. His clean example is the email phrase, “I hope you stay in good health and condition”—understandable when translated word for word, but culturally characteristic of English rather than ordinary Polish correspondence.
Labenz presses whether this is merely cultural idiosyncrasy or a measurable reasoning gap. Kozlowski’s answer is that common benchmarks reveal too little: multiple-choice tasks test recognition, extraction, or summarization, but rarely the fluency and structure of long-form generation.
3. Polish competence requires history and idiom, not vocabulary alone
The PLCC, or Polish Linguistic and Cultural Competence benchmark, was created to test linguistic and cultural competence across grammar, vocabulary, tradition, history, art, entertainment, and culturally typical expressions. It also probes ambiguous Polish words and whether phrases are used in their proper context.
Kozlowski concedes that even PLCC does not yet adequately measure long sentences and extended Polish structures. His warning—“we are biased by the benchmarks”—is that a strong multiple-choice score can conceal generation that no native speaker would naturally write.
His central formulation is that “language is not only the wording.” A native-like system must mix grammatical control with idioms, cultural references, real places, historical context, and the judgment to deploy each in the appropriate situation.
4. PLLuM’s recipe makes post-training data the real moat
Kozlowski explains training through a school analogy: pre-training teaches words, structures, and pieces of information; supervised fine-tuning teaches tasks such as essays, summaries, or calculations; preference learning supplies the marks that guide correction.
The final stage can use reinforcement learning with human feedback, DPO, or ORPO, but the underlying logic is the same. Pre-training creates language capacity, SFT builds competencies, and preference feedback shapes which answers the model should repeat or avoid.
Synthetic instructions can weaken a model when their language is poor, so PLLuM emphasizes organic instructions and preferences or puts machine-generated examples through human editing. Kozlowski attributes part of frontier labs’ quality to large stores of manual instruction data that remain proprietary.
PLLuM published an almost 100-page paper or “cookbook,” plus samples of its datasets, instructions, and preferences on Hugging Face. The aim is reproducibility beyond weights: “open source is not only about open weights.” The team also found that models aligned for English speakers could be more easily hacked than models aligned for Polish speakers, requiring local security work.
5. Poland postpones an AI constitution because data law bites first
Labenz asks how a national model chooses among contested values, including whether Poland’s historically Catholic majority should produce Catholic AI. Kozlowski notes that religious identification varies between cities and villages and has been changing. PLLuM currently targets broadly ethical behavior, avoiding hate speech and behavior that could be unethical or rude, without an elaborate political or religious constitution.
“We have much more liberty now,” he says while explaining that PLLuM releases base, instruct, and chat models. Because alignment constraints can disrupt business cases, companies can choose among different model types rather than receiving one heavily constrained system.
Kozlowski expects a more explicit constitution might emerge in one or two years, but Poland’s immediate constraints concern what data may legally be used. The EU AI Act and Polish authors’ rights rules can have a harder quality impact than behavioral principles because they may exclude “80% of the data” from training datasets.
6. Europe curates a smaller corpus while US crawlers capture much of it
Kozlowski says PLLuM has several hundred billion tokens overall, with around 200 billion remaining after deduplication and filtering. Web data remains the majority, because repeated pages, malformed text, special characters, and low-quality fragments remove a large share of any crawl.
Kozlowski says Polish sites observed “plenty of Anthropic crawlers,” including on pages whose robot rules disallowed them. His practical point is jurisdictional: a Polish rights holder may possess evidence yet lack the resources to challenge a well-funded US company in a US court.
Non-web supplements include libraries, scientific literature, publications, and bilateral agreements with publishers for material not openly posted. Even so, Kozlowski believes OpenAI or Anthropic may already possess 80–90% of the Polish data available to PLLuM.
The more distinctive supply comes after pre-training: dozens or hundreds of internal annotators use PLLuM’s own tools to write and edit instructions and preferences. Cooperation or licensing with other players may eventually be necessary because “there is a place where you are not able to go further.”
7. On-premise agents shift the advantage toward small specialists
Kozlowski predicts that “the agentic AI revolution will be based on small, localized models.” Regulated companies and public agencies often cannot send sensitive work to cloud APIs, making on-premise inference, local GPUs, energy consumption, and controllability central design constraints.
Buyers rarely need a ChatGPT-like system covering 1,000 tasks. They commonly have 10 or 20 use cases; once they calculate the cost of perhaps 16 GPUs and the electricity to run them, they seek the smallest textual or multimodal model that clears the required performance threshold.
His deployment rule of thumb is at least 1,000 organic or human-reviewed instructions for a task set, with more preferable. Supervised fine-tuning at that scale can deliver the same—and sometimes higher—quality as a giant cloud model prompted zero- or few-shot.
Labenz agrees for controlled workflows but highlights the convenience of frontier APIs. Kozlowski’s rebuttal is operational: cloud systems offer broad capability out of the box, while private systems trade dataset preparation for controllability, data locality, lower deployment cost, and reduced dependence on cloud vendors.
8. Frontier releases can improve globally while regressing in Polish
Kozlowski says a Claude release declined on his team’s Polish linguistic and cultural benchmark; he reports similar issues with GPT models. Some GPT releases have not improved in Polish competence, and some have become worse.
If vendors prioritize software-development assistance because it is a core paid use case, capacity devoted to niche-language writing may fall. A Polish organization can therefore integrate a model successfully, then discover months later that the next release no longer meets its established requirements.
Labenz finds the claim striking: frontier labs might decline excellent Polish data if they prefer spending training capacity on synthetic coding tasks. Kozlowski accepts data cooperation as a natural future step, while emphasizing that foreign vendors may simply regard Poland as a non-core market.
9. Public deployment matters more than winning model market share
PLLuM includes not just weights but assistants and chatbots intended for citizens, city inhabitants, and municipal offices. Kozlowski argues that the practical bottleneck may be “there are no chatbots for cities,” not whether a benchmark-leading model gains another marginal point.
The initiative is funded by Poland’s Ministry of Digital Affairs and organized as a public consortium, initially involving six and later eight institutes and universities. That structure changes the objective function: the team is not primarily optimizing customer count, subscriptions, or conventional return on investment.
Its stated priorities are openness, regulatory compliance, transparency, organic data, security, and broad public-sector usability—especially through closed, on-premise deployments. Models remain an asset, but value is realized only after customization for actual citizen and administrative workflows.
10. Continued pre-training substitutes for an unaffordable Polish foundation run
PLLuM tried to create models from random weights, but Kozlowski says an 8-billion-parameter model requires at least 1 trillion tokens for stable, high-quality pre-training. Poland’s roughly 200 billion curated tokens were insufficient to create a moderate- or good-enough-quality model from scratch.
The team instead starts from Llama and Mistral base models and performs language adaptation: a few epochs of continued pre-training over Polish text, followed by SFT and preference optimization. It is also testing multilingual mixtures to reach the required token scale for a random-weight run.
Continual learning introduces forgetting. Some other language abilities decline after Polish adaptation, but PLLuM does not deliberately prune them; enough prior knowledge persists for the model to continue generating English, while general knowledge is generally retained.
Time-sensitive facts remain a separate problem. A model trained through March 2025 cannot memorize later legal changes, accidents, or political appointments, so Kozlowski favors retrieval from an updated knowledge base and uses the localized model to synthesize the retrieved evidence.
11. Domain adaptation works only after enterprises confront their data
PLLuM performed continued pre-training for PKO, which Kozlowski describes as the largest bank in Central and Eastern Europe. Using the bank’s closed domain corpus produced large quality gains on some financial tasks, though improvements varied by evaluation.
His threshold is roughly 10 billion tokens after deduplication and filtering, implying perhaps 30–40 billion raw tokens. He doubts a 40-person company’s Slack, Google Docs, Jira tickets, proposals, and revision histories would reliably survive processing at that scale: “It’s not so easy.”
He estimates that fewer than 100 European companies may have 100 billion tokens of closed internal data. Even large organizations must exclude sensitive or legally ambiguous emails, cybersecurity material, intellectual property, and other data with unclear permissions.
Kozlowski’s sequencing is categorical: inventory and clean the repositories first, classify quality and rights, then train or deploy AI. Companies often begin integration without knowing what data they own; those projects “usually collapse” because curation was treated as an afterthought.
12. Cost discipline challenges frontier scaling, but does not settle the race
Kozlowski cites Llama 3.4 and Kimia among models whose licenses, he says, prohibit EU use, tentatively linking the restriction to general-purpose-model documentation required under the AI Act’s second chapter, released in August 2025. He says builders must disclose training data, security steps, datasets, and resources that some providers may not wish to reveal.
He would consider Chinese models for bounded extraction or document analysis, where censorship risk is low, but not readily for essays, histories, or long emails where censored assumptions become more noticeable. Model origin matters differently depending on whether the task is analytical or generative.
Poland is focused mostly on AI factories, while ministry funding for a consortium supports compute, model development, and talent. Kozlowski says Poland cannot match US compensation, where he has heard that elite researchers receive contracts “like the quarterbacks in the NFL.” He sees China and the United States as the two dominant players, says Europe lacks a comparably strong player, and mentions Mistral while reporting that he heard 30–40% of its shares are held by Microsoft.
Kozlowski sees GPT-5’s improvement over GPT-4 as steadier and more “horizontal” than the shocks of ChatGPT in 2022, GPT-4 in 2023, and multimodality in 2024. Labenz keeps the disagreement alive: $10 billion runs might still decisively beat $1 billion runs, while Kozlowski insists most businesses should benchmark the email, extraction, or cross-selling task they actually need.
His geopolitical tension is equally unresolved: federated European collaboration distributes knowledge and power, but fast products usually require centralized money, compute, and talent. Federated learning may eventually unlock secure shared repositories, yet firms still cannot value their own data; it is “not for the next year,” even as the internet’s organic-data reservoir appears “almost full.”