SCROLL NEWS / DISCOVERY
Search headlines
Find the stories shaping the conversation.
Results for " Reprints " 9 found 🔀 AI-powered Shuffle
Wizards of the Coast Confirms Reprints for Popular MTG Warhammer 40K Commander Decks
In a statement to IGN, Wizards of the Coast has confirmed a new "small" reprint run for the incredibly popular MTG Warhammer 40K Commander Decks.
AI Assistant Tool QED Scores bioRxiv Preprint Quality
QED, an AI assistant tool, evaluates the originality and validity of bioRxiv preprints, assigning them QED Scores. Researchers report that its rankings often align with expert opinion.
Scientists ejected from diabetes conference for distributing journal reprints
Those ousted included ADA journal editor-in-chief Steven Kahn and former ADA president Desmond Schatz...
NASA, Schmidt Sciences give $7M boost to arXiv
Cornell Tech just landed more than $7 million from NASA and Schmidt Sciences to give arXiv — the massive open-access research hub — a serious upgrade. The money will help arXiv modernize its infrastructure, finish moving to the cloud, and build smarter tools to help researchers find preprints more easily. “I am profoundly grateful for… MoreNASA, Schmidt Sciences give $7M boost to arXiv
QED Science: Interview With VP Of Product Guy Ronen About Validating Scientific Research With AI
QED Science provides an AI validity engine that breaks manuscripts and grant proposals into individual claims, tests each against the published literature, and scores them for originality and validity. The company also publishes The 1%, a blind ranking of the strongest new preprints in life science. Pulse 2.0 interviewed QED Science VP of Product Guy Ronen to learn more.
Flash Promos & Origins Reprint Updates
Learn about Flash Promos for stores, and updates on Origins and Proving Grounds reprints.
“AI polls” are fake polls
A few weeks after Donald Trump’s second presidential win, I took the train up from London (where I was living at the time) to Oxford to attend a conference on polls and forecasts of the 2024 election. Most of the attendees were pollsters or academics, but I also watched presentations from Aaru and Electric Twin, two companies that do what is interchangeably called synthetic sampling, silicon sampling, or synthetic audiences. Stripped of startup jargon, that means they use large language models (LLMs) to simulate responses to public opinion polls by having AI agents take on the role of survey respondents. I had already heard of Aaru thanks to some articles with eye-catching headlines like “No people, no problem: AI chatbots predict elections better than humans” in the months leading up to Election Day. The founders were making some big, some might even say far-fetched claims, such as: “within two years, we will simulate the entire globe — from the way crops are grown in Ukraine to how that impacts production of oil in Iraq, trade through the strait of Malacca, and elections for the mayor of Baltimore.” When Semafor asked Aaru’s cofounders — Cameron Fink and Ned Koh — about my boss, they said “we respect all those who came before us.” Nate (as he so often does) shared his thoughts on Twitter: Fink and Koh were relatively good-natured about this back-and-forth when we spoke at Oxford. They even offered to mail me one of the t-shirts featuring Nate’s quote they apparently had made. I never took them up on the offer, which I now somewhat regret. These synthetic sampling companies fell off my radar for a while, but they do still exist. In fact, Aaru recently received a $1 billion valuation. Is what they’re doing anywhere close to the most important frontier in AI development? Not by a longshot, especially when Anthropic just developed a model so adept at exploiting software vulnerabilities that it’s only being released to 40 companies. Still, silicon sampling is increasingly finding its way into public polling. Axios reported in March that “a majority of people trust their own doctors and nurses” based on findings from Aaru — without mentioning that the “people” in that sentence were actually LLMs. Around the same time, the Public Sentiment Institute “boosted” their online sample of 373 real survey respondents with 114 AI agents.1 (Spoiler alert: even the co-founder of Electric Twin doesn’t think that’s a particularly defensible approach.) Polling companies like Qualtrics and Ipsos are also developing synthetic data panels. So, what should we make of these … “polls”? Let’s get one thing out of the way: whatever they are, they’re not polls in the way that term is usually defined. Subscribe You can’t replace polls with AI On one hand, using LLMs to essentially make up fake survey respondents sort of sounds like the dumbest idea ever, one that will at best imperfectly replicate real polls while introducing all sorts of biases. On the other hand, with LLMs improving at a remarkable, perhaps even alarming rate, maybe that means I’m a dinosaur at the ripe old age of 24 because I still want to rely on polls that talk to actual people. I’m not going to argue that synthetic samples are completely useless. In fact, as I’ll return to later, there is evidence that some techniques can replicate topline survey results quickly and cheaply. But the marketing from certain companies can be slightly optimistic. “No traditional poll will exist by the time the next general election occurs,” said Fink in 2024. We’re just 206 days away from the midterms, and based on the fact that I still have to collect a bunch of polls every day, I’d say he should have run that prediction by a sample of AI agents before the interview.2 To see why synthetic samples can’t replace polls, here’s a quick primer on how they work. The simplest version of these models involves taking a LLM (like ChatGPT or Claude), giving it a demographic profile (e.g., a white, college-educated woman who lives in Utah and makes $70k a year), and then asking it to respond to a survey question. You repeat that process a few thousand times using different demographic profiles and end up with a sample of synthetic survey responses. The actual models used by private companies are more sophisticated than this, usually because they incorporate more hypothesized demographic characteristics for each agent and provide them with extra information. Aaru, for example, feeds agents a diet of news and information they’d be likely to consume, while Electric Twin incorporates their customers’ proprietary data about the audience they’re trying to replicate. The way Ben Warner, the co-founder of Electric Twin, explained it to me was “we have a large amount of data on […] for instance, 5,000 people. Can we make an accurate prediction of how they would respond to another question?” Still, it should be obvious why synthetic samples can’t replace polls. Polling is fundamentally a data collection process. We might use surveys to make predictions by feeding them into election forecasts, but the main purpose of a poll isn’t prediction, it’s gathering new data about what people think and how they feel. Silicon sampling, on the other hand, produces no new data. It’s simply a model: you input LLM training data, demographic prompts, and a bunch of other information, and it spits out a prediction for what a poll would say. We love models here, but models aren’t polls. That difference is an important philosophical sticking point for most pollsters I talk to. “I think politics should stay away from [synthetic sampling], because we’re trying to […] represent the voice of the people,” said Natalie Jackson, a vice president at GQR Insights. Democratic pollster John Hagner told me: “I think I’m just incredibly skeptical of this idea. I don’t think it’s research. At that point, you’re asking the machine to tell you what you already believe.” Hagner has seen some presentations of early synthetic sampling experiments, but so far, “if it’s being used in a campaign, people are keeping it incredibly quiet.”3 But Eli, I hear you saying, aren’t polls themselves increasingly governed by modeling decisions? Indeed they are: pollsters’ choices on which sampling method to use, how to define their likely voter models, and how to weight their samples can and do lead to dramatic differences in the results they publish. Aaru even referenced these limitations in the methodology statement included with that maternal mortality “poll” — although I’m using the term “methodology statement” loosely here, because it doesn’t really explain how the model works at all. We can ignore the (frankly preposterous) implication that synthetic sampling isn’t subject to a separate set of biases. The important point is that there’s still a meaningful difference between using weighting and other statistical techniques on actual polling data and using a model to predict what a poll would say. The latter is far closer to election forecasts or techniques like MRP — potentially useful models, but not a replacement for polls.4 To be fair, other synthetic sampling companies are perfectly happy with the distinction between polls and models. Warner compared polling and synthetic sampling to different tools in a toolbox. “The mistake I think we make is we think that these new tools should either work in exactly the same way or somehow replace these old tools,” he said. “Rather than thinking of it as, okay, so we’ve always had the hammer, we’ve always had the screwdriver, now we’ve got a saw. But don’t use a saw to try [to] do the job of a hammer.” A quick comment from Nate Eli didn’t ask me for a comment — rather rude of him, don’t you think? But since I’m editing this story, I figured I’d add a few quick thoughts rather than putting words in his mouth. Beyond the frequently misleading marketing, what bothers me about the AI “poll” hype is that as AI tools make statistical inference cheaper and/or better (note that these are not synonyms) that actually increases the comparative value of collecting original data. You might be able to train a model to make a reasonable estimate of what some hard-to-reach poll respondent would say — say, a young Black man who voted for Trump. (Such a person checks a number of boxes for a voter who is usually hard to reach in surveys.) Indeed, this is closely related to what models like the Silver Bulletin forecast already do. They essentially smooth out the kinks in noisy survey data by making inferences based on past voting patterns or national polls or surveys of other states. But you don’t actually know what these voters think unless you’re reaching them directly. If there’s a shift in opinion among this subgroup, you’re not going to detect it. So if I were running a campaign, I’d invest more in going the extra mile to find a representative sample of those voters. And then I’d hire some smart quants — or Claude? — to figure out the implications for campaign strategy based on proprietary data that my competitors didn’t have access to. -Nate Silver Are these models any good? If synthetic surveys are just a new type of model, the next obvious question is whether the models are at least accurate. The answer very much depends on who you ask. On one end of the spectrum, you have the maximalist argument that synthetic sampling is more accurate than actual polls. “It’s an incredibly challenging problem to go to someone and say ‘hey, we’re going to be more accurate at predicting human behavior than you, even when you talk to your customers directly’,” Koh recently told CNBC. In his view, synthetic sampling isn’t a saw to polling’s hammer, it’s “magic.” There’s certainly evidence that synthetic samples can replicate certain survey toplines. But if Aaru does have any examples of their approach outperforming the polls, they’re keeping those to themselves.5 Aaru’s 2024 election model, for example, had Kamala Harris leading in Michigan, Nevada, Pennsylvania, and Wisconsin on November 4th. And although they’ve since taken down their forecast page, they gave Harris a 50.5 percent chance of winning the race on November 2nd.6 After the election, Fink told Semafor he was happy enough with those results because they were “within margin of error,” a term that is completely meaningless when applied to a “sample” of AI agents. And of course, Aaru says their models have improved since 2024, so supposedly now they’d be more accurate than the polls? Still, their stronger argument is on cost: “We are significantly faster and cheaper than traditional polling, and still more accurate,” said Fink. The first two claims are undeniably true, but the third brings us to the opposite end of the spectrum. Both Jackson and Hagner are skeptical that these models are reliable for anything beyond replicating common survey toplines. “I just […] don’t think the machines are what we want when we’re looking for nuanced views. My example on this is people in Arizona and Nevada in 2024 who voted for Trump and voted for expanding abortion in their states on ballot initiatives,” said Jackson. Hagner identified another issue. Maybe the synthetic respondents, like sycophantic LLMs, are inhuman in one important way: they’re too nice. “The reports that have come through at the meetings that I’ve been at are that the early experiments on this, they cannot get respondents to be as racist or sexist or, frankly, as negative as human respondents,” he said. Academic research mostly agrees on this point. While there are some papers that show promising results when using LLMs to replicate polling data, most show that LLMs suffer from various quirks like producing too few “don’t know” responses and can seriously overpredict the favorability of politicians like Donald Trump and Kamala Harris. They also seem to struggle with too little variation between demographic subgroups, so the difference in predicted opinion between Democrats and Republicans, for example, is too small. When I asked Warner about these studies, his response to these papers was that just because academics can’t get synthetic sampling to work doesn’t mean that the technique doesn’t work in general. “Actually, the argument is, okay, yours does not [work]. That does not mean […] for this complex set of machinery, which uses a lot of investment, a lot of time, a lot of money, you can’t get it to work.” Cards on the table, I’m somewhat sympathetic to this argument because academics aren’t exactly great at making election forecasts. Usually, the people with skin in the game are the most accurate. Warner’s argument is that the approach Electric Twin takes — which includes, for example, making multiple predictions for each synthetic respondent using different models and prompts and subsequently averaging those to get a final prediction in a sort of ensemble forecast — produces better results than the simpler academic models. Warner shared a comparison between his method and the method from a recent academic paper with me, and Electric Twin was indeed able to get more accurate replication. But even still, he acknowledged that synthetic sampling “is not a crystal ball.” “If you asked me, do I think using other data sources will be more accurate than asking somebody who they will vote for, I would probably say no. But if you asked me ‘would your system be useful for our turnout modeling today?’ I would say yes.” For better or worse, it looks like the method is already getting more popular in the market research world. Most of the clients Aaru touts these days are businesses like EY and McDonald’s. And AI will probably begin to pop up in other parts of the political polling process. Pollsters are already using it to code open-ended survey responses, and some firms, like YouGov, are testing using LLMs to ask survey respondents questions. More worryingly, one danger to actual polls is that AI agents can be used to infiltrate online surveys. Most online polls use various checks to prevent that from happening, but there’s conflicting evidence on how effective those filters are and how prevalent AI agents currently are in online panels. If those agents ever become impossible to detect, it might spell the end of online polling, but the solution isn’t to replace all of your respondents with ChatGPT. Silver Bulletin is a reader-supported publication. To receive new posts and support our work, consider becoming a subscriber. Subscribe 1 That particular poll obviously doesn’t meet Silver Bulletin standards for aggregation. But we exclude all Public Sentiment Institute polls from our averages because we classify them as an amateur polling firm. 2 You could argue that Fink meant the next presidential election, but (a) I’m also confident we’ll still have real polls in 2028 and (b) in that case he should have asked an LLM to define “general election.” 3 Quick caveat: that’s reporting from a Democratic pollster. It’s possible that Republicans are more willing to use AI in political campaigns. 4 Indeed, Silver Bulletin does not include “polls” produced by MRP in our forecasts or averages, and we think it’s extremely misleading when their practitioners describe them in a way that suggests original data had been collected among a large number of states or Congressional districts. 5 A recent report from Aaru and EY did show two examples of a synthetic estimate being closer than a survey to a real-world benchmark — but I’d take those findings with a grain of salt because the report reads more like an ad and didn’t involve any sort of prediction being made ahead of time. 6 For comparison, our odds for Harris on the same day were 48.2 percent.
The Science Behind the AI Panic Is Shakier Than You Think
A column by Sascha Lobo Is AI making us dumb? So asked DER SPIEGEL on its cover a while ago. After a few weeks of recovery and a therapeutic intervention with the help of my AI, I feel able to respond. That’s a bit unfair, of course; the article itself is more nuanced than the loud cover, the image of a schoolkid with dull prompt questions, or the subtitle: “AI is conquering the schools, and educators fear a disaster.” The short answer to “Is AI making us dumb?” is: no. The long answer is more interesting, and even carries a preliminary glimmer of the spectacular. Unfortunately, it requires a detour into the sphere of artificial intelligence, into the mogul-field terrain of technology assessment, the attention economy, and the destructive virus of cultural pessimism. Along with narrowing the scope to AI’s effects on education and work rather than on intelligence. Because the honest version of the question would be: Does AI help us learn and work, or not? Generative AI, which set off the still-ongoing hype with ChatGPT, has only existed in relevant distribution since late 2022, early 2023. For a technology-driven transformation of people and society, that is extremely short. For that reason alone, research is still in its infancy. And technology assessment is not a fast discipline anyway. Prominent voices from Elizabeth Eisenstein to RAND to Tom Wheeler say we still cannot conclusively judge the consequences of the printing press. Its invention is a mere 580 years back, and some scholars would rather not circulate assessments that might have to be retracted in two or three hundred years. We’re Going Through Growing Pains A phenomenon that already caused upheaval in social media research makes serious assessment of AI’s consequences even harder: the “moving target” problem of LLM-ology, as the linguist Sean Trott called it in 2025. AI develops so quickly and so unpredictably that researchers often end up studying snapshots of the past. There are concepts, from morphological analysis to anticipatory governance, for keeping research, planning, and regulation halfway meaningful anyway. But even with those, universally valid statements are rare. DER SPIEGEL - The German View is a reader-supported publication. To receive new posts and support our work, consider becoming a free or paid subscriber. Subscribe We should therefore treat findings about the interplay between humans and AI as far more procedural and situational. Unfortunately, this challenge comes at the worst possible time. We are going through growing pains: AI’s everyday impact has grown very large very fast, and with it the longing for guidance. Annoyingly, the body of solid knowledge has not exploded along with it. Experts cannot be moved to broad consensus on even banal statements about AI. Sascha Lobo, born in 1975, is an author and strategy consultant focusing on the internet and digital technologies. Together with Jule Lobo, he explores the debates of the day in the podcast “Feel the News – Was Deutschland bewegt” (”What Moves Germany”). Nvidia CEO Jensen Huang says software developers are dying out. Bill Gates considers coders one of only three professions that will survive the AI tsunami. Top tech economists find virtually irrefutable arguments both for and against an AI bubble. The same naturally applies to the structurally youth-hostile debate over whether AI is good or bad for young people. The gap between the demand for insight and the supply of solid knowledge creates an orientation vacuum. This vacuum of AI cluelessness is filled, as the attention economy described by Georg Franck in 1998 would have it, with half-knowledge (funnily enough, often AI-generated), marketing, and self-marketing. After all, infinite money is at stake, along with academia’s best current shot at global fame. But this iridescent froth of interpretation operates more by the algorithmic principles of the digital public sphere than by seriously earned insight. That is why dramatized, fear-feeding claims about AI are so promising. It tempts parts of academia into distorted drivel, and parts of the media then sensationalize the published drivel once more. Was the Research Aimed at a Desired Result? How badly this can influence academia and public perception is demonstrated by one of the most-discussed studies on AI’s effects: “Your Brain on ChatGPT.” The MIT study by Nataliya Kosmyna also gets plenty of space in the SPIEGEL cover story. What was measured and interpreted: lower brain activity in people who write with AI assistance, compared to people who only google or write without any tools. As is common with preprints, the study has not yet been peer-reviewed, and it contains substantial weaknesses that experts have sharply criticized. The first round had 54 participants in three groups, the second only 18, who reportedly also came from MIT’s own orbit. There was no preregistration (committing in advance to what you want to find out with which data). In their analysis, Stanković et al. even discovered an opposite effect that the study does not further explain. Some measurement data has not (yet) been published. And the EEG analysis method used is considered too imprecise to support such far-reaching conclusions. The biggest criticism, though, is the suspicion of an activism-driven study. In an interview with Time, lead author Kosmyna said she published the study immediately, rather than after full peer review, because she feared politicians might otherwise decide to put ChatGPT in kindergartens. She considers that absolutely harmful. It is legitimate to hold such a view. It just seems unscientific to approach a study with it, and then to design the setup so that the chance of a fitting desired result remains as high as possible. Ideologies Collide It is not easy to separate genuine AI insights from studies that trivialize and weaken well-founded AI criticism. The same goes for the opposing, AI-enthusiastic narratives, because the pro-AI side works against it with billions in resources. Ideologies collide, fed by egos, PR strategies, moral posturing, and once-in-a-lifetime opportunities. Doomsday marketing makes the search even harder: AI companies can profit from warnings about the dangers of AI, whether through the notoriety of apocalypse narratives or because, in some target groups, danger translates into power. Some seemingly negative findings are therefore quite welcome in the AI industry. If you try to get at the findings presumably less influenced by interest groups, the picture in education seems relatively clear: AI can improve educational outcomes when the right AI is used the right way, namely pedagogically. Simply dumping an AI chatbot into a classroom, by contrast, accomplishes little. Huge surprise. But something else is also obvious: whether they are allowed to or not, kids use AI for school, which is why the controlled integration of AI tools into education is, in the medium term, entirely without alternative. In the world of work, the landscape of findings is more complex and requires more differentiation. A series of serious studies points to an AI effect that suits neither the alarmists, regulation fetishists, and AI opponents, nor the trillion-dollar corporations with their armada of paid scientists, PR professionals, and ecosystems: working with AI helps the inexperienced (at least for now) considerably more than it helps the pros, in certain fields of work, and provided they know how to use it correctly. Missing Domain Knowledge Among the experienced, the professionals, and the high flyers, by contrast, despite frequent time savings, partially negative effects and new problems are emerging. Take the jagged frontier problem: AI solves task A brilliantly and fails completely at task B, even though both look similar even to professionals. Or the verification-cost problem, in which AI’s greater productivity gets eaten up by the enormous effort of checking its output. Or the illusion of competence, because AI makes new domains seem easily penetrable, until you painfully discover that deep domain knowledge is missing. Taken as a whole, the effects of working with AI do lean positive. But the promise that AI works as a turbocharger for everyone and boosts practically everything simply refuses to materialize. The AI magic the world has been feeling since ChatGPT arrived in late 2022 has so far barely shown up in most companies’ numbers. The consulting firm McKinsey has coined a term for this, the “gen AI paradox,” and believes that work processes and corporate structures would have to be rebuilt quite fundamentally to realize AI’s benefits at scale. Reuters just reported that only 3.3 percent of Microsoft’s customers have subscribed to the AI package Copilot. That may be down to the product; in other areas we continue to see soaring successes and new AI triumphs. But after the first quarter of 2026, disillusionment is the word for all too many AI projects at traditional companies: because the promised revolution of AI agents has so far failed to arrive or been postponed, and because of the leveling effect that many of the studies cited here suggest. Against Inequality The societal effects, on the other hand, appear rather encouraging. Many previous technologies operated on the Matthew principle, named for the Gospel verse about giving to those who already have: they tended to reward the more competent and more privileged. AI, however, seems more often to have an equalizing effect once a threshold of access and education is crossed. Some anecdotal insights point in this direction, for example, how an unemployed man in Leipzig successfully defended himself in court in 2025 against fraud accusations from Germany’s federal employment agency — without a lawyer, armed only with ChatGPT. The AI made mistakes and cited invented court rulings, but in the end the man walked away unpunished. And a key capability of artificial intelligence is its capacity to learn: set up correctly and fed with data, it can keep getting better. It is a question of time, or more precisely of AI innovations and training data, until even free AI chatbots write better legal briefs than law graduates with perfect exam scores. Whether that will make anyone dumb is something the world’s press will surely inform us of in due time. Subscribe Leave a comment
QED Science Launches AI Infrastructure for Scientific Validation
Recognizes The 1% After Blind Evaluation of 57,455 Scientific PreprintsTEL AVIV, Israel, June 24, 2026 (GLOBE NEWSWIRE) -- QED Science, a science validity engine and modern infrastructure for researchers, funders, institutions and life science organizations today launched The 1%, the top 574 life science preprints. The company anonymously scored 57,455 bioRxiv preprints submitted between May 2025 and April 2026 for originality and validity. The ranking is powered by QED Score, QED Science’s vali