15.S07, Week 1, The Bakeoff

I am, week by week, going through Pierre Azoulay’s course on innovation.

Last week, I read “The Bakeoff,” and live-tweeted it.  Here are some extended thoughts:

At its core, this article is about a noble experiment by a naive operator (Gundrum), trying to answer the question: what is the optimal way we should form teams, in order to innovate?

I like Gundrum, because hey, he invented Mrs. Fields, but I call him naive because the 3 forms of team were kind of haphazardly thrown together.  In particular, one team is in traditional hierarchical style, and the other two are inspired by software coding practices (XP, or pair programming, and open source).

Besides the first, which is a control, I really don’t think the other two represent distinct “methods of innovation” as Gundrum intends.  Which makes sense — neither pair programming nor open source were developed in order to innovate faster or better, per se.

What would we like to see instead?  Well, just think about any good experiment.  What we want to do is a) have a theory about some independent variable X that we think will influence our rate/quality of innovation, and b) set up our teams such that, as much as possible, all that is different between them is a randomized tweak of X.  Obviously you can’t literally clone a team and do exactly that, but it’s the form we want to approximate, to say anything about X’s effect on Y (innovation).

There are of course other ways to set up the experiment (e.g., you could have one team, and introduce a plausibly exogenous shock — Z — into the system that only acts on Y through X), but basically we are after isolating effects.

I actually thought Gundrum did a reasonably good job of thinking about how to measure Y, though, after all — a representative sample engaged in a fair taste test, and voted on which cookie they liked best.  This is a not-bad, if maybe high-cost and slow, way to measure how “good” at innovating each team was (although of course, innovation != tastes best, necessarily…).

Anyway a good first exercise, and helpful in prodding us to think about how we might set up a better experiment, if we were in Gundrum’s shoes.


Annotated: “Academic signaling and the post-truth world”

Wherein I annotate things.

Today, responding to (the more fun half of) Noah Smith’s blog post, “Academic signaling and the post-truth world”:

Lots of people are freaking out about the “post-truth world” and the “war on science“. People are blaming Trump, but I think Trump is just a symptom.

For one thing, rising distrust of science long predates the current political climate; conservative rejection of climate science is a decades-old phenomenon. It’s natural for people to want to disbelieve scientific results that would lead to them making less money. And there’s always a tribal element to the arguments over how to use scientific results; conservatives accurately perceive that people who hate capitalism tend to over-emphasize scientific results that imply capitalism is fundamentally destructive.

But I think things are worse now than before. The right’s distrust of science has reached knee-jerk levels. And on the left, more seem willing to embrace things like anti-vax, and to be overly skeptical of scientific results saying GMOs are safe.

I’m choosing to skip over this bit, because many reasons, but mostly it just wouldn’t be fun for me.

Read More »

Monday Night Science & Innovation Links, December 20, 2016

Got catching up to do!  More links!

“This is just a pointer to two new (non-technical) papers of mine that look at the implications of various falling costs associated with new technologies.” — Digitopoly | Falling Costs: Two Non-Technical Papers

“Those departures put pressure on Alphabet to transform its science project into a working commercial product.” — Google is launching a new self-driving car company called Waymo – Vox

“So, to sum up: They aren’t privy to his data. He isn’t privy to them. And because they work from encrypted data, they can’t use their machine learning models on other data—and neither can he. But Craib believes the blind can lead the blind to a better hedge fund.” — Numerai Used 7,500 Faceless Coders Paid in Bitcoin to Build Its Hedge Fund’s Brain | WIRED

“If a thousand virtual worlds take shape, so too can a thousand AIs.” — Google’s Improbable Deal to Recreate the Real World in VR | WIRED

“Interested readers can view our complete recommendations, but a new Trump national space policy should declare:[…]” — Opinion: Dear President Trump: Here’s How to Make Space Great Again | WIRED

“Are Ideas Getting Harder to Find? Yes, say Bloom, Jones, Van Reenen, and Webb.” — A Very Depressing Paper on the Great Stagnation – Marginal REVOLUTION

Sunday Night Science & Innovation Links, December 19, 2016

Annnnd we’re back.  Finals is a pain.

“Using data on expenditure on research and development, and patent applications, receipts, and citations, we show that the Chinese economy has become increasingly innovative. ” — From “Made in China” to “Innovated in China”: Necessity, Prospect, and Challenges

“Yes, it may be a damaging four years for research, innovation, the economy (driven by R&D), and the environment – some irrevocable. But that’s not reason to lose hope. Instead it’s a challenge to all of us to get involved. We must be more dedicated than ever to work for change.” — Dear Scientists: Our Government Needs You – Scientific American Blog Network

“Science, rather than appearing like a human enterprise, full of fits and starts in the never-ending search for knowledge, is expected to prove claims once a week, or even more frequently. And I think that’s bad for readers and viewers.” — Why science news embargoes are bad for the public – Vox

“This means the social costs of new techniques (as opposed to the costs captured in market prices) are systematically underestimated.” — Bite-back, Joel Mokyr

“What will happen to those efforts under a Donald Trump presidency? One thing seems likely: Set aside Mars. Private companies are going to get a chance to do business on the moon.” — What a Trump presidency means for NASA and the future of space exploration — Quartz

Sunday Night Science & Innovation Links, Nov 27, 2016

A lot of people talk about science and innovation.  Few of them talk to each other.  

Talk!  To each other!

“By moving research closer to the team that actually builds the products, the company believes it can develop a better understanding of how AI can do things customers truly want.” — Artificial Intelligence Is Driving Huge Changes at Google, Facebook, and Microsoft | WIRED

“If anything, the situation for underrepresented minorities is getting worse. Gibbs found that between 2005 and 2013, almost 6,000 scientists from those groups earned their doctoral degrees, while the number of assistant professors fell by six.” — The Minority Talent Pool in Science Is Draining Away – The Atlantic

“But as Richard Smith, the former editor of the BMJ, summed up: “We have little or no evidence that peer review ‘works,’ but we have lots of evidence of its downside.”” — This new study may explain why peer review in science often fails – Vox

“Carafano’s recent report on science policy argues that OSTP should be eliminated because it is unnecessary given the other sources of science advice the president can access” — Advocate for Eliminating OSTP Appointed to Trump Transition Team | American Institute of Physics

“A move like this, if it actually happened, could be a big deal. Not only would it mean serious changes to US climate research, but it could affect a host of other key NASA programs that provide info on everything from weather to wildfires to drought and much more.” — A Trump adviser wants to scale back NASA’s ability to study climate change – Vox

“[the plan] shows what the NIH is interested in and (likely) where grants will follow. And that could ultimately shape the direction of behavioral and social science itself.” — Social Science Is Busted. But the NIH Has a Plan that Could Fix It | WIRED

“Because of the competition for grants, scientists often propose projects that they know will work rather than on ideas that are more adventurous with potentially greater payoffs.” — James Simons’s Foundation Starts New Institute for Computing, Big Data – The New York Times

Sunday Night Science & Innovation Links, Nov 20, 2016

A lot of people talk about science and innovation.  Few of them talk to each other.  

Talk!  To each other!

“‘Even as an anti-establishment president elect, he is not going to thumb his nose at the public. The public understands the value of science and technology. At least it does sometimes.'” — Scientists Prepare to Fight for Their Funding Under Trump | WIRED

“A new “science of science policy” is emerging, and it may offer more compelling guidance for policy decisions and for more credible advocacy.” — Wanted: Better Benchmarks | Science

“IRIS is creating a data platform incorporating an array of metrics on the science and innovation system, from individual participation and career trajectories, to material and equipment purchasing patterns, to publication and patent outputs over the long term. — A Q&A with the Institute for Research on Innovation & Science | AAAS – The World’s Largest General Scientific Society

“Not everyone can go out and grab thirty AI-happy astrophysicists.” — Giant Corporations Are Hoarding the World’s AI Talent | WIRED

“Once prediction became cheap, innovators reframed driving as a prediction problem. Rather than programing endless if-then-else statements, they instead simply asked the AI to predict: “What would a human driver do?”” — Digitopoly | The Simple Economics of Machine Intelligence

“the hype around machine intelligence methods continues to grow: the words “deep learning” now equally represent a series of meaningful breakthroughs (wonderful) but also a hyped phrase like “big data” (not so good!).” — The current state of machine intelligence 3.0 – O’Reilly Media

“They often share a common ideology, tied not just to the neoliberal drive to privatize, innovate and disrupt, but to long-standing modernist ideas about creative destruction, quantification and the value of scientificity.” — What Is A Media Lab?