Consider the Experiment

We're proposing a new scientific asset: the experiment. Creating an alternative asset class for scientists helps diversify income opportunities, improve research outcomes and could catalyze a shift away from the over-emphasis on published papers. If done well, it would improve communication and transparency of the full scientific process, especially around costs. Most importantly, it would encourage more researchers to pursue high-risk, high-reward ideas at the earliest stages. 

In the spirit of prototyping what we preach, we're launching the first Experiment Patron NFT, tied to a project studying deep sea materials science. All proceeds raised in excess of the project costs (including carbon offsets for the NFT) will be donated to the Experiment Foundation to fund more scientists. 

Outline:
1. Background
2. Experiment as Asset
3. The Two Paths
4. Incentivizing Open Science
+ Acknowledgements

“No one would become a scientist solely for the money. There are too many other, more lucrative careers that require fewer years of training and fewer hours of work and pay higher salaries. Nonetheless, success in science is accompanied by monetary rewards, and scientists are not immune to their allures.”

- Paula Stephan, How Economics Shapes Science

Economics shapes science. Many scientists are driven by idealistic curiosity and the path up the scientific career ladder is by community validation, but as in most professions, they’re ultimately paid in cash. There is renewed interest in examining scientific incentive structures. The leading researchers in the science of science funding have emphasized the importance of thinking about the "portfolio" of scientific investment. Basically, they’re concerned with how we diversify investments between grants, fellowships and tools to maximize the opportunity for new knowledge and discovery. This has resulted in passionate debates about funding people versus projects, compelling ideas for new types of research organizations, and better definitions of risk in science. The portfolio approach has largely been discussed in terms of how to allocate government-sized budgets, but it could also be assessed from the perspective of individual scientists. How are they investing their precious time and energy? How does the current reward system shape their priorities?

The current funding system is dominated by one metric: papers. The peer-reviewed paper is the current standard for scientific knowledge and researchers are heavily incentivized to publish. Bibliometric measures — impact factors of journals or the number of citations — have become the de facto measure for scientific reputation and the rewards associated. Altmetrics (alternative metrics) have emerged to try to capture a wider perspective of importance, including things like PDF downloads and Wikipedia references, but these scores have had limited influence on the economics of science and the dominant role of the paper. 

The basic lesson from Modern Portfolio Theory (MPT) is that diversification across asset classes helps reduce risk and increase returns. Theoretically, the same could hold true for a range of scientific assets. An important outcome of MPT has been the formalization and broad acceptance of alternative financial assets — real estate, venture capital, fine art, cryptocurrency, etc — even at the level of the retail investor. The definition of alternative assets is simply that they’re not stocks, bonds, or cash — the “standard” asset classes. The value and prices of alternative assets usually has a low correlation to standard assets, which makes them a useful tool for diversification. This got me thinking: What are (or could be) alternative scientific assets? What vehicles exist (or could exist) that move inversely or independently from bibliometrics?

Of the money-making opportunities for scientists, two options stand out as true alternative scientific assets: startups and science communication (like popular science books or YouTube channels). Both are good for science, eliciting different behaviors from scientists than the rote publishing hustle and providing avenues for people from non-academic backgrounds to contribute to the scientific project. By opting into the startup track, scientists can increase the translational impact of their work by effectively building a commercial offering for their ideas. The economics, and the project’s corresponding focus, are tied to the success or failure of the company. Scientist-founded startups bring more investment capital into research and development, and they maintain an orientation towards future real world products and impact. Science communication efforts have similar upsides. The media business has its own economic metrics. Views, sales, and ratings determine who gets paid. This drives more experimentation around the methods of communicating science to a broader audience. These alternative assets are net improvements to science because they make it more robust. They diversify income opportunities, the behavior of scientists, and research outcomes.

We should be bold and thoughtful with the invention of new asset classes that speak to our highest aspirations. The rise of crypto economics has demonstrated a new way to define, create, and exchange value. Scientists should seize the moment.

There are currently two discussions happening at the intersection of cryptoeconomics and science. The first is a surface level discussion: Universities auctioning off remnants of famous intellectual property as well as important criticism of corresponding energy use. But there’s also a much richer conversation happening below the surface. Small groups and nascent teams are grasping the potential of these tools as an opportunity to invent new ways for scientists to pursue their curiosity and they’re building out the ideas. Several of these decentralized science (or “DeSci”) visions stand out as notable. Matt Stephenson’s ideas about how NFTs can create a sort of alt-VC model. Or Brian Armstrong’s vision of a tokenized replacement for journals. More than just ideas, they’re being prototyped upon conception — it’s a thrilling time to imagine and build something new. This trial and error will reveal the path forward.

Like every good citizen of this emerging DeSci scene, we aim to test and learn in public, too. Here’s our idea:

Consider the experiment, the engine of empirical progress. For many researchers, being at the bench or in the field is a deeply rewarding source of joy. It's the thrill of standing at the edge of what is known and peering beyond. Humans advance our collective knowledge one falsifiable test at a time, yet, for most people outside of academia, the process of experimentation is one of the least visible parts of the scientific process. Sharing the experiment is an important new frontier for open science. 

Now consider Experiment. The website is a crowdfunding service for scientists, but it's also more than that. The design of the Experiment page has brought two previously private documents, the grant proposal and the lab notebook, online, and included a dynamic way to involve a community of supporters. It's a funding mechanism, but it's also a new form of open science communication. It's scholarly work, too. Each project is assigned a Digital Object Identifier (DOI) number so the pages themselves can be easily cited for academic reference. Thousands of researchers have used the site to put their ideas to the test. Asking for support and validation in public is difficult and requires a high degree of conviction. The ideas — and the scientists — become all the better for having done it. 

There's another aspect of Experiment that is worth emphasizing: open proposals provide transparency around the costs of scientific research. This is uncommon yet important. The costs of doing research impact what gets studied, whether that’s determining the size of a telescope to build or deciding whether to use male or female mice. Costs drive decisions about what equipment a lab will buy or share. They also affect the pace of discovery. The dropping costs of genetic sequencing, for example, have created an explosion of research. 

The Experiment model creates downward pressure on the costs of doing science. Instead of budgets inflating or squeezing to fit grant limits, Experiment budgets are scoped to fit each project precisely. There's an incentive to be lean, because it equates directly to higher likelihoods of funding success. 

Lower costs don't mean cheap science. In fact, there is compelling evidence that science funding is undervaluing the high volume, small-first funding strategies employed by venture capitalists and angel investors in the financial world. Transparent and competitive budgets are an important component of encouraging this type of risk-seeking investment in science.

Experiment has honed the design of open grant proposals and lab notes through the forge of crowdfunding, but it's well positioned to go a step further. I've watched with interest as Non-Fungible Tokens (NFTs), blockchain-verified digital assets, have sparked the imagination and creativity of artists and patrons around the world. Beneath the hype and criticism, there's something genuine and important happening in terms of rethinking how we value digital goods. It's time for science to start exploring this new tool. And the experiment is the perfect unit to build on. 

For the first time, we're offering a Patron NFT for an Experiment: What technology survives at the deepest part of the ocean? 

It will operate much like any other project on Experiment, meaning backers will be able to donate to gain access to future Lab Notes and updates. But there will also be a single Patron NFT offered on OpenSea. Purchasing the Patron NFT is different from making a regular pledge, as the NFT holder may re-sell this token on the open NFT market. The Patron will not own the right to any intellectual property beyond the NFT (although that's an idea worth testing, and VitaDAO are trying it for longevity drugs), so it's better to think of these as a sort of scientist trading cards, or like buying Charles Darwin’s notes as a collectors item.

Adding the Patron NFT does not change the nature of the question-asking or workflow on Experiment. It does, however, create a verifiable certificate of impact. This is new and potentially useful. It gives us a mechanism to value scientific ideas through time. There is a rich history of thinking on “idea futures” by Robin Hanson and others. Running this test on Experiment — turning the open proposal plus lab notes into a unit of tradable value — opens the door to new possibilities. Whether companies, larger science funders, or NFT collectors will create a secondary market for these assets remains an open question. I hope making it tangible and real forces the conversation.

To illustrate how Experiments could alter the trajectory of ideas and redefine the scientific process, let's play out our project on two possible paths: the traditional route and the new, open route. We'll do this making the assumption that, if we makes a useful new discovery, we would like some combination of recognition and financial compensation in addition to the satisfaction of contributing to the ongoing pursuit of knowledge.

We’re raising money to build a large high-pressure chamber to test materials at the equivalent of full-ocean depth pressure. From the project page: 

What is the context of this research?

The pressure in the deep ocean is astounding and few conventional designs can withstand the force. Understanding the effects of tremendous pressure on various off-the-shelf engineering components and materials is critical for designing novel systems that can operate at that great depth without the need for protection from pressure-proof chambers. To conduct these tests, a small pressure chamber (50mm internal diameter) capable of reaching pressures over 1100 bar (16000 PSI) was designed and constructed. Preliminary results suggest that more things can survive at these pressures than one might expect. The design of a larger chamber will allow us to test more components with more complexity.

What is the significance of this project?

The deep ocean is the largest and most unexplored habitat on our planet. It plays a critical role in regulating climate, is a large potential source for marine resources, and is a vast home to unique biodiverse creatures. The ecosystem is currently vulnerable to unexpected changes due to climate change and the potential for deep sea mining is growing. Understanding and monitoring these ecosystems is more important than ever.

There is a scientific urgency to exploring the deep ocean, yet it remains a challenge. Expanding the knowledge base of materials and engineering components capable of performing at such depth would accelerate the potential of robotic systems to explore the deep ocean.

Like all great experiments, the outcomes of this research is unknown. Whether we take the traditional or open path, the project starts the same way: by raising enough money to purchase materials and build the test chamber. 

If this project were to proceed down the traditional path of research and development, we would fill out a grant proposal similar to the project proposal we have submitted on Experiment, although some foundations or federal agencies require much more extensive paperwork. The added paperwork makes sense if we're talking about allocating millions of dollars, especially for taxpayer-funded federal research, but can be cumbersome for individual researchers seeking relatively small funding amounts. To make matters worse, we need to apply to several potential funding groups to ensure something will come through. Each one has unique application requirements and processes. We would spend time formatting, adjusting, and adding to each proposal while minding a handful of timelines and submission deadlines. Then wait. Similar to the application requirements, the timelines for reviews and decision making are all different and dictated by the funder to streamline their process. This lack of standardization makes perfect sense from the perspective of the funders, but creates a cumulatively significant cognitive burden for potential grantees. 

The applications often happen in waves: letters of intent, abstracts, and full proposals. If successful, proposals are followed by coordination with funders’ legal and finance teams to complete grant paperwork and set up payments. Again, this makes sense for large grant proposals and projects that will allocate resources for years in advance, but it seems excessive for our little experiment. It's rare that this process, from inspiration to funding, takes less than six months. If we were doing this through a university, the institution might add an additional 50% of the cost as overhead.

Compare this to the open route. We filled out the proposal on Experiment in a few hours, submit it for review, and have content edits and feedback on scientific legibility in less than a week. The project goes live on the site and opens for fundraising. We can accumulate donations through the crowdfunding process, but can also attract a single funder through the Patron NFT sale. If the project is successfully funded, it's rare that the process from inspiration to funding would take more than six weeks. 

Ok. Now say fundraising was a success and the money is in the bank. It's time to do the work. The pressure testing chamber is built. The testing begins. 

On the traditional path, we would take meticulous notes in lab notebook to ensure maximum credit in future publications and patent filings. If the results aren't interesting, that's likely the end of the process. We send in the required reports to our funders to keep them apprised, and move onto new things. The test chamber either stays in the garage until inspiration strikes again, or if testing was done through a University lab, the tooling stays with them. The details of the project fade to obscurity, shutting the door for others to build on our learnings. 

But suppose we do make a breakthrough discovery! Material X turns out to perform exceptionally well at full ocean depth and this new knowledge will change how ocean robotic systems are designed forever! This is valuable knowledge. We double down on documentation. We bring in colleagues to witness and sign the lab notebook for proof, backing up the case for a patent. Then get to work writing up the results. To get academic credit, we need to be the first to publish these results in a peer-reviewed journal. We quickly get a manuscript together, writing up methods and outcomes. Some of this is a re-articulation of everything in the proposal, but it now includes experimental observations, results, and a new conclusion. 

The publishing process takes more time. After waiting to hear back from journal editors to see if the paper will move to the next step, the peer-review process begins. A handful of reviewers take months to provide feedback and suggestions, until finally the paper is ready for publication. This is valuable and can often push the work forward, but it can also take a lot of time. Preprint servers have improved and sped up the feedback mechanism, but we still need the peer-reviewed publication. Finding the right journal and getting reviewed and edited can take years. At the end of this process, if the journal isn’t open access, we may have to sign away copyright and ownership of the paper. But we will get credit. 

We also want to make sure this idea gets beyond academic circles and out into the world. If there's a financial opportunity, we hope it provides us with enough compensation to continue with our research and development, supporting our mission to explore the depths of the world's oceans. So we file a patent on the same day that we submit the manuscript. This requires the help of a patent attorney and could cost several thousand dollars. If we’ve done this through a university, we’ve also just signed the destiny of the intellectual property over to a (hopefully capable) technology transfer office. Now we have to put together a business plan that convinces investors of the promise and potential of the innovation as well as the market for deep ocean exploration. We offer the investors a big chunk of the company to help make this real. This process is fraught with risk and uncertainty. If this becomes a commercial success, the financial windfall is probably a decade away. Add another twenty years to the date of the patent and that's when the rest of the world can freely and creatively use this new knowledge. 

The open path is very different. It starts with a sale of the Patron NFT on Experiment or a handful of donations from friends and fans. The newly-formed team begins a journey of discovery. 

We document just as meticulously as before, but now the updates are in the public Lab Notes section of the Experiment. Backers, fans, and onlookers can comment and suggest iterations and ideas to improve the tests. The dialogue is documented and public so everyone can follow the discussion. We have every reason to communicate the ideas and learnings effectively, because the value of the experiment grows with the science and the story. The Experiment project has a DOI number so the project can be easily cited in future academic work. We’re getting credit for the work we’ve done as we do it. 

If the initial ideas don't generate any positive results, that doesn't mean the project is a failure. The null results are cleanly and clearly documented for any future researcher or funder to learn from and to build on. The experiment also doesn't have to end there, because we are doing this work with a community of supporters. We can articulate an iteration of the initial test that uses new materials, perhaps from a suggestion from the community. The original Patron might agree to sell the NFT to a new Patron, and because of the way we've structured the NFT contract, the original holder is partially paid back and we get enough of the new funds to continue testing. The story grows with every new Lab Note and documentation of what doesn't work. The value accrues according to a faithful execution of the scientific method, as opposed to the results alone. 

What about in the case of the breakthrough discovery of Material X? Again, we still double down on meticulous notes, continuing to publish them on Experiment, but this time we aren’t doing it with a patent in mind. We want others to replicate the results and use Material X as soon as possible. We also create new "Patron or Backer Only" content that provides even more in-depth documentation conveying the tacit knowledge needed to implement this new material in projects. We’re less worried about filing a patent, instead using an open source license that ensures attribution to this Experiment. 

The Patron is pleased. They are now the holder of an increasingly valuable piece of scientific history. They field offers from ocean robotics companies who want access to the in-depth notes and collectors who want to add to their science NFT portfolio. The Patron holder gets an offer they like and sells. We’d get a meaningful percentage of that sale, just as we will for the continued lifetime of this experiment and Patron NFT. The Experiment is an enduring asset that pays dividends. We could still publish a peer-reviewed paper on the results, but we’d have to justify the investment of time and the logic of signing any rights over to a journal. The work is being cited. We stand to continue to gain financially. And we’re already thinking about the next experiment.