An overview of the Climate Finance Tracker, an initiative launched with OneEarth. Each circle or node on the map represents one private company in the U.S. Users can sort by themes, sectors, investors, and more.

The emerging threat of climate change to our national security

From as early as 2008, the U.S. Department of Defense flagged the increased risk to national security if climate change wasn’t urgently addressed by the federal government.

“We judge global climate change will have wide-ranging implications for U.S. national security interests over the next 20 years,” wrote the National Intelligence Committee during President George W. Bush’s tenure.  The political will to take action to address climate change as a national security issue is slowly growing. Recently the U.S. Army released its first ever climate strategy, outlining its goals to cut emissions in half by 2030, produce electric combat vehicles, and advance officer training for a warmer climate. 

At Primer, we build AI technology that helps subject matter experts and decision-makers identify, understand, and act on emerging threats. We believe that the intersection between climate change and national security is imperative to proactively address. 

“Climate change will affect the Department of Defense’s ability to defend the nation and poses immediate risks to U.S. national security.”

-U.S. Department of Defense, 2014 Climate Change Adaption Roadmap

“Increasing migration and urbanization of populations are also further straining the capacities of governments around the world and are likely to result in further fracturing of societies, potentially creating breeding grounds for radicalization.”

-U.S. National Intelligence Strategy, 2019

Our partnership with OneEarth.org

On the cusp of Climate Week in New York starting Sept. 19, Primer is proud to announce our partnership with OneEarth.org, and the public release of the Climate Finance Tracker that we worked on together this year. Leveraging Primer’s machine learning technology on publicly available data, the tracker “follows the money,” surfacing in one unified view, where U.S. private funding for combating climate change is going. In short, following the money helps everyone collaborate more effectively to tackle the climate crisis. 

For the first time, the Climate Finance Tracker is live and open-source, and uses AI to advance intelligence and decision cycles, to ultimately improve outcomes on solving a major global challenge.  

The Climate Finance Tracker allows anyone to search the funding landscape of all private companies and nonprofits in the U.S. that are addressing climate-related topics. By surfacing topics from organization and project descriptions provided by Crunchbase and Candid, the tracker allows us to better understand the shape of the current response, and with that, its potential gaps. This understanding means funders can better direct their resources, ensuring the highest possible impact of their investment.

The value of following the funding

Eric Berlow, a leading social impact data scientist and researcher, led the project with his nonprofit Vibrant Data Labs and OneEarth.org. Berlow explained how using Primer’s AI technology allows researchers to find the answer to questions they may not even know to ask. 

“We searched for companies working on known climate-related topics, and then the machine pulled out of their language that some also mention national security-related topics,” Berlow explained. “That’s novel. I wouldn’t have even known to look… And the machine found it.”

Text-based machine learning doesn’t just uncover answers – it also helps surface questions researchers may not even know to ask.

A snapshot of 32 organizations that were tagged with ’national security’, ‘unrest, conflicts and war’, or ‘peace and security.’ Explore the full Climate Finance Tracker here.

“We know climate intersects with national security, and now we can easily see about 25 private investors funding companies in that space,” Berlow explained. “It’s only 6 companies out of 6,000. That’s an important gap. Let’s talk to them and see what’s going on.”

Climate week begins

With this week’s launch of the Climate Finance Tracker, along with Climate Week in New York, we expect to see increased conversation around adoption of machine learning solutions to help solve global problems such as the climate crisis, making us more resilient toward emerging threats.

At Primer, we believe that when we empower the ecosystem with better information, intelligence, and decision making, we can truly solve really complex problems together. 

See the power of text-based AI and NLP. 

Request a demo of Primer’s NLP technology. 

Primer partnered with Vibrant Data Labs, a nonprofit that uses data to create the first-ever comprehensive map of climate change funding. Our interview with Eric Berlow of Vibrant Data Labs shows how ‘following the money’ reveals both our biggest opportunities and threats to turn climate change around.

It’s no secret that the future of our planet is in trouble. The recently reported IPCC report concluded that countries aren’t doing nearly enough to protect against significant disasters to come. 

But in order to solve a big problem like climate change, and to understand if our current response is working, we need to see where private funding in the sector is going. What issues are getting money, and which organizations are getting that funding? What other trends might emerge? 

Applying NLP to climate change

That’s where natural language processing comes in. Using Primer’s NLP technology, we partnered with Eric Berlow at Vibrant Data Labs to produce the first-ever climate change funding map. Primer’s Engines analyzed data on over 12,000 companies and nonprofits funded in the last five years. Using organizations’ descriptions provided by Crunchbase, grant applications provided by Candid, and in partnership with the Cisco Foundation, we generated one of the first-ever data-driven hierarchies of climate topics to better understand our current response, alongside any potential gaps. Using this topic hierarchy, we can see what projects organizations are working on – and where. That helps us see what’s missing in the bigger picture. And to solve a problem like climate change, a big picture view is what’s needed.

Watch our interview with Eric Berlow on why following the funding is crucial for the climate’s future. 



“The Coronavirus pandemic was like the trailer to the climate change movie where if the have-nots are really impacted, everybody gets affected by it. Climate change is one of those problems that is an all-hands on deck problem. You cannot just solve a little piece of it. “ – Eric Berlow



Learn more about Primer’s partnership with Vibrant Data Labs here, and learn the technical piece behind the work here and here.

For more information about Primer and to access product demos, contact Primer here.

Copy a simple Apps Script into any Google spreadsheet to quickly run multiple Primer Models on text in any data cell

Understanding a large corpus of text-based documents is getting a lot easier with NLP. Where previously we might need a human to perform a manual review of social media posts, news articles, or financial and legal documents, now we can process them using NLP models to extract entities and key phrases, understand sentiment, and classify documents.

This tutorial shows you how to run a sentiment model and a key phrase extractor with Primer Engines to analyze customer reviews. We’ll get our reviews from Amazon and then use Primer’s pre-trained models (we call them Engines) to analyze sentiment and extract phrases from each review to see what patterns we can detect. To make things even simpler, this tutorial lets you play around with Engines inside of Google Sheets.

To get your Engines API key, Sign up here: https://primer.ai/engines-sign-up.

Once you have an account you’ll have the ability to create your api key within the app:
https://developers.primer.ai/account/api-keys

All you’ll need to do to run this tutorial is have a Google spreadsheet and to sign up for free for Primer Engines. Let’s get started. 

Set-up

To start, let’s collect the top 10 customer Amazon reviews from this Bubble machine, as well as a few one star reviews. You can do this yourself, but we’ve gone ahead and collected them in this spreadsheet that you can clone. 

Once we have built our dataset, we want to empower the spreadsheet with NLP. To do this I’m just going to copy this snippet of code into the Apps Script module of the Extensions tab.

var api_key_row = 2;
var api_key_col = 2;
var API_KEY = SpreadsheetApp.getActiveSheet().getRange(api_key_row, api_key_col).getValue();
 
// flag for editing without triggering requests.
let calls_disabled = false
 
// default Engines request data
class EnginesRequest {
 constructor(endpoint) {
   this.base_url = “https://engines.primer.ai/api/”;
   this.endpoint = endpoint
   this.headers = {“Authorization”: “Bearer ” + API_KEY,”Content-Type”: “application/json”};
 }
 
 post(json_body_as_string) {
   // Fastest option but subject to 30/minute Engines rate limiting
   if (calls_disabled) {
   return “Engine Offline”
   }
   let request_url = this.base_url + this.endpoint;
   let options = {“method”:’post’,’payload’:json_body_as_string,’headers’:this.headers};
   let response = UrlFetchApp.fetch(request_url,options) ;
   return [[this.endpoint,response.getContentText()]]
 }
 
 post_async(json_body_as_string) {
   if (calls_disabled) {
   return “Engine Offline”
   }
   let request_url = this.base_url + this.endpoint;
   let options = {“method”:”post”,’payload’:json_body_as_string,’headers’:this.headers};
   let response = UrlFetchApp.fetch(request_url,options) ;
   if (response.getResponseCode() !== 200) {
     return [[this.endpoint,response.getContentText()]]
   }
 
   let result_id = response.getContentText();
   Utilities.sleep(200);
   options = {“method”:”get”, ‘headers’:this.headers};
   let get_url = this.base_url + “v1/result/” + result_id.replace(/[‘”]+/g, ”);
   let get_response = UrlFetchApp.fetch(get_url,options);
   let count = 0;
   while (get_response.getResponseCode() == 202 && count < 1000) {
     Utilities.sleep(1000);
     count += 1;
     get_response = UrlFetchApp.fetch(get_url,options);
   }
   return [[this.endpoint,get_response.getContentText()]]
 }
}
 
function genericEngineSync(endpoint, text) {
 if (text === “” || text === null || text === undefined) {
   return “Engine Disabled – Text Field Required”
 }
 let req = new EnginesRequest(endpoint);
 let body = JSON.stringify({ text });
 return req.post(body)
}
 
function genericEngine(endpoint, text) {
 // By default generic engine is async as synchronous requests are rate limited.
 if (typeof API_KEY === ‘undefined’) {
     return “Engine Disabled – API Key Required”;
 }
 if (text === “” || text === null || text === undefined) {
   return “Engine Disabled – Text Field Required”
 }
 let req = new EnginesRequest(endpoint);
 let body = JSON.stringify({‘text’ : [text]});
 return req.post_async(body)
}
var api_key_row = 2;
var api_key_col = 2;
var API_KEY = SpreadsheetApp.getActiveSheet().getRange(api_key_row, api_key_col).getValue();
Once you’ve pasted in the data be sure to save the script.

Extracting sentiment with Primer’s NLP models

Now that we have the logic in place that we’ll call from our spreadsheet. Let’s give it a try with the Primer’s Sentiment model (For a full list of models available, check out the API documents here: https://engines.primer.ai).

In our sheet let’s put a cell next to the first review and use following function call:

=genericEngine(“v1/classify/sentiment”,A4)

Our cell will have an error stating we need to include the API key.

You can update the script to have the key defined, or you can update location the script is currently looking for the key: Column B, Row 2.

To get your Engines API key, if you haven’t already: Sign up here: https://primer.ai/engines-sign-up. Once you have an account you’ll have the ability to create your api key within the app:
https://developers.primer.ai/account/api-keys

With the key added in – and partially obscured 🙂 – you should now see the results displayed for your input data. A model’s magnitude, 0.97 in this case, shows the model’s confidence in this sentiment label. The closer to 1, the higher the confidence in the label. For full details on our Sentiment model, please see the documentation here: https://developers.primer.ai/docs#v1-classify-sentiment

You can now start getting a quick look at the data by dragging the function to include our other data cells.

Looking at the data manually, it looks like the bubble maker wasn’t a positive experience for one cat owner. We can run the Primer Key Phrase extractor model on the data to get additional details that may help us understand why. 

Just add the following code next to that cell and let’s take a look at phrases in this negative review.

=genericEngine(“v1/generate/phrases”,A8)

Some keywords come to the surface: “motor”, “noisy”, “scares” that give us an idea of where this negativity is coming from. The span field shows us the exact character index the phrase was found in the text! Let’s add this code to the whole sheet so we’re both determining sentiment for every review and also extracting phrases to give us a dataset that we can parse and display for some high level insight.

Ok, we’ve extracted the sentiment and key phrases of the document. To make the data easier to parse, we can copy the following bit of JavaScript Apps Script we copied over. It will add some formatting logic.


class DefaultDict {
 constructor(defaultVal) {
   return new Proxy({}, {
     get: (target, name) => target[name] || defaultVal
   })
 }
}
 
function aggregatePhraseSentiment(sentiment_range, phrase_range) {
 if (typeof API_KEY === ‘undefined’) {
     return “aggregatePhraseSentiment script”;
 }
 let positive_sentiment_phrases = new DefaultDict(0);
 let negative_sentiment_phrases = new DefaultDict(0);
 let phrase_seen_count = new DefaultDict(0); // for sort by value descending to show most popular phrase
 let phrase_name = “”
 for (const phrase_list_index in phrase_range) {
   phrase_dict = JSON.parse(phrase_range[phrase_list_index]);
   if (“phrases” in phrase_dict) {
     for (phrase_index in phrase_dict[“phrases”]) {
       phrase_name = phrase_dict[“phrases”][phrase_index][“phrase”]
       phrase_seen_count[phrase_name]++;
       // check corresponding sentiment e.g. {“sentiment”: “negative”, “magnitude”: 0.973}
       sentiment_dict = JSON.parse(sentiment_range[phrase_list_index])
       if (“negative” === sentiment_dict[“sentiment”]) {
         negative_sentiment_phrases[phrase_name]++;
       }
       else if (“positive” === sentiment_dict[“sentiment”]) {
         positive_sentiment_phrases[phrase_name]++;
     }
   }
   }
 }
 
 var popular_phrases_descending = Object.keys(phrase_seen_count).map(function(key) {
   return [key, phrase_seen_count[key]];
 });
 
 popular_phrases_descending.sort(function(first, second) {
   return second[1] – first[1];
 });
 results = [[“aggregatePhraseSentiment script”, “Phrase”, “Count Positive”, “Count Negative”]];
 for (phrase in popular_phrases_descending) {
   key = popular_phrases_descending[phrase][0];
   count_positive = positive_sentiment_phrases[key];
   count_negative = negative_sentiment_phrases[key];
   results.push([“”, key, count_positive, count_negative]);
 }
 return results;
}

Now from our sample set reviews, we can see at a glance which features were positive and which were negative about the bubble maker. You can add as many reviews into this dataset as you’d like to understand the customer experience of a particular product. Or you could do this with any unstructured dataset for your organization. 

All Primer Engines can be called using the Google Sheet Apps Script function calls. Here’s a collection of a few Primer Engines being run on sample data.

+Link to the Primer Engines in Google spreadsheet.

For more information about Primer and to access product demos, contact Primer here.

From a security perspective, the Beijing Olympics has all the ingredients of a perfect storm. Global tensions are ratcheted up as the US and Russia negotiations about Ukraine’s fate hit a wall. The US and its allies have declared a diplomatic boycott of the Beijing Olympics due to alleged human rights violations by the Chinese government against the Uyghur population. These same dissidents are looking at the games as a platform for voicing their grievances. 

Adding to the tensions are reports that movements of people in the city, including players and media, are limited under COVID-19 protections.  The games themselves are expected to be more opaque than years prior as Beijing has limited foreign correspondents to a “closed loop” bubble. The correspondents are provided with limited interactions with athletes taking part in the games and no movement in the city itself. The athletes are under extra security protocols as officials warn of Chinese surveillance operations that will target them while in the country. 

Security analysts from global security operations centers (GSOCs) around the world with assets or people in Beijing during the games are on high alert. They need to monitor incoming information around the clock to anticipate any threats and provide instructions if any security incidents occur.

Surface key insights

For security teams charged with the herculean task of monitoring threats emanating from and against the Beijing Olympics, and its commercial sponsors, Primer Command® is a game changer.  Command not only identifies people and places mentioned, but it also shows live feeds of news and social media posts side by side streaming in. This saves analysts from using multiple apps — it’s all in a single pane. Further, this allows them to leverage news reports to corroborate, in seconds, social media posts with alleged threats and emerging issues of importance. With these capabilities in hand, organizations can maximize the safety of their people, operations, and assets. 

Primer Command automatically structures and summarizes millions of news and social media posts across 100 languages to surface key people, locations, organizations, topics, suspected disinformation, sentiment, and media to better understand how the situation is evolving. 

The following puts a spotlight on the power of Command’s ability to zero in on the information that matters most through advanced filters and AI-enabled classifiers. To learn more about Command’s other capabilities, click here for a free trial of Command for Tactical Insights.

Humanitarian aid filters

Command’s advanced filtering capabilities allow security teams and first responders to unlock mission-critical information during a crisis. Primer’s humanitarian aid filters include drilling down on tweets of displaced people and requests for evacuations, caution and advice, and reports of injuries or deceased people. These filters will be particularly operative during the winter games to zero-in on any violence and safety concerns for personnel there.

Chinese language filters

Learning what is being conveyed to the local population will be more important during the Beijing games given the limited movements allowed for foreign media outlets. Additionally, it will give security teams early indicators of unrest. This filter can also illuminate posts by local nationals – automatically translated – expressing concern about physical security threats.

Additional filters

Command can filter on numerous other entities to drill down into the information security analysts care about most.  

  • Event Types: Security analysts can filter the information feeds by the event types such as diplomacy, military, or law enforcement topics within news reports. This will prove to be of particular importance if any security incidents break-out during the winter games. Analysts will be able to home in on reporting related to law enforcement to get the latest actions to contain the threat. Focusing on these posts also provides GSOCs with the latest official statements and guidance by security forces for the people in the area. 
  • Disinformation: Command can detect and flag disputed information in news reports. Analysts can filter by disputed information and use this as an indicator of disinformation campaigns occurring during the events. 
  • Social Data: Analysts can segment social media data feeds based on the number of likes or retweets, key threat indicators, or even by sentiment. Primer’s state-of-the art sentiment filters are hand tuned for the highest accuracy so analysts can quickly identify the social media posts that matter. By filtering for negative sentiment, analysts can uncover the threats hidden within the deluge of data — separating chatter from hazards.

Learn more

Contact sales to discuss your specific needs. You can also stay connected on Linkedin and Twitter.

What is NLP and how can it support business?

Imagine a curious executive walking down a path that explores how Natural Language Processing (NLP) can help their business by uncovering insights hidden in the available and pertinent data to make better and more timely decisions. The executive is often faced with the problem of having vast amounts of company data, and not a lot of ways to take action on it. Not to mention the risk of not knowing what’s hiding in the data.

Steps one and two

As the journey begins, the executive takes steps one and two, Ideate and Identify. The executive asks “What do I want to know?” and follows up with “Where do I find the answers?” Whether it is customer attitudes toward a business, how it compares with competitors, or almost anything else decision-makers, analysts and owners would want to know, a quest for knowledge is the beginning. The next consideration is where to find answers to these questions. Identifying those data sources — internal purchase information, call center logs, product descriptions and reviews, social media posts, customer survey results, etc. — is the “where” that the answers will be found.

Steps three and four

The next steps, three and four, are Connect & Ingest and Transform, where the executive might ask, “How do I find the answers?” extracting text from both external sources and the company’s unstructured internal data mentioned in step two (Identify). In Transform, the executive asks, “How do I use NLP and AI?” focusing on Named Entity Recognition (NER), a sub-task of information extraction. It identifies and classifies named entities mentioned in unstructured data into predefined categories such as given and surnames, affiliated organizations, geographic locations, time expressions, etc. It also includes question & answer, classification, topic modeling, relationship extraction, sentiment analysis and other methods of processing the ingested information into something useful.


Steps five and six

Next up, Integrate and Explore, steps five and six, come after data has been scanned and processed. At the Integrate step, the executive could ask, “How do I combine insights from NLP and AI with my own data and analytical models?” To sharpen the results of NLP, companies often have pre-existing internal mathematical models, analytics and projections that can be combined. Once completed, the executive at the Explore step asks, “What answers do I have?” and looks at the patterns and unearthed relationships that can be converted into action plans.

Steps seven and eight

Operationalize and Realize & Repeat are steps seven and eight. Once the executive has answers from previous steps, the question is “How can I use this information?” The Operationalize step adds these new insights into a workflow. This can include replacing labor-intensive and often mundane tasks like manually compiling mass volumes of data with automation, contextual routing, summarized analysis, and creating intelligence dashboards. 

The last step is what keeps the process going, which seems counter-intuitive, but the executive learns it is a feature of NLP. Once new insights are put in place to realize achievable outcomes, this new data is used to expand on and repeat for further analysis that will result in a deeper understanding.

While these concepts require a basic understanding of NLP, the eight steps succinctly sum up the process. The executive has developed a better understanding of how NLP can positively impact their bottom line. 

Primer strives to help the world understand the power of NLP and what it can do to help businesses make better decisions and gain a competitive advantage. The “8 Steps to Get Started with NLP” is one of myriad efforts to pique interest, start conversations, and educate the business community.

For more information about Primer and to access product demos, contact Primer here.

The NLP revolution kicked off in 2018 when BERT dropped. In 2019 and 2020, it was one mind-bending breakthrough after another. Generative language models began writing long-form text on demand with human-level fluency. Our benchmarks of machine intelligence — SNLI, SQuAD, GLUE — got clobbered. Many wondered if we were measuring the wrong things with the wrong data. (Spoiler: yes.) Suddenly, we no longer needed tens of thousands of training examples to get decent performance on an NLP task. We could pull it off with mere hundreds. And for some NLP tasks, no training data was needed at all. Zero-shot was suddenly a thing.

Over the past year, NLP has cooled down. Transformer architecture, self-supervised pre-training on internet text — it seems like this paradigm is here to stay. Welcome to the normal science phase of NLP. So, what now?

Time to industrialize

Efficiency, efficiency, efficiency, and more efficiency. The cost of training and serving giant language models is outstripping Moore’s law. So far, the bigger the model, the better the performance. But if you can’t fine-tune these models on fresh data for new NLP tasks, then they are only as good as your prompt engineering. (And trust me, prompt “engineering” is more art than science.) The focus on efficiency is all about industrializing NLP.

“The revolution, in the style of Kazimir Malevich” by John Bohannon using CLIP+VQGAN

The NLP benchmarks crisis is another sign that we’re in the normal science phase. Here’s an open dirty secret: The purpose of today’s NLP benchmarks is to publish papers, not to make progress toward practical NLP. We are getting newer, better benchmarks — Big-Bench, MMMLU — but we likely need an entirely different approach. The Dynabench effort is an interesting twist. But until we stop using performance metrics known to be broken — looking at you, ROUGE — industrial NLP will be harder to achieve.

Normal science is applied science. We are moving from “Wow, look at the impressively fluent garbage text my model generates!” to “Wow, I just solved a million-dollar NLP problem for a customer in 2 sprints!”

NLP is eating ML

But meanwhile, another revolution is brewing on the edges of NLP. The phrase “Software is eating the world” became “ML is eating software.” Now it looks like “NLP is eating ML,” starting with computer vision.

The first nibble was DALL-E. Remember when you first saw those AI-generated avocado armchairs? That was cool. But then along came CLIP+VQGAN. And just last month, GLIDE made it possible to iteratively edit images generated by text prompts with yet more text prompts. This could be the future of Photoshop.

The trend is the merging of cognitive modalities into a single latent space. Today, it’s language and photos. Soon, it will be language and video. Why not joint representation of language, vision, and bodily movement? Imagine generating a dancing robot from a text prompt. That’s coming.

Go beyond English

Bilingual language models lost the throne in 2021. For the first time, a multilingual model beat them all at WMT. That was a big moment that surprised no one. Machine translation has caught up with the multi-task trend seen across the rest of NLP. Learning one task often helps a model do others, because no task in NLP is truly orthogonal to the rest.

The bigger deal is the explosion in non-English NLP. Chinese is on top — no surprise given the massive investment. There are now multiple Chinese language models as big or bigger than GPT-3, including Yuan and WuDao. But we also have a Korean language model, along with a Korean version of GLUE (benchmark problems notwithstanding). And most exciting of all is an entirely new approach to multilingual NLP called XLS-R trained on a massive collection of speech recordings in 128 languages. This could become the foundation NLP model for low-resource languages.

Build responsibly

If 2020 was the year that NLP ethics blew up, then 2021 was the year that ethics considerations became not just normal but expected. NeurIPS led the way with ethics guidelines and a call on authors to include a “broader impact” section in their papers. 

But like the rest of the AI industry, NLP is still self-regulating. That will change. The most significant action in AI self-regulation last year was Facebook’s voluntary deletion of its facial recognition data. That move was surely made in anticipation of government regulation and liability. (They also have more than enough surveillance data on their customers to power their targeted advertising.)

Where will the rubber meet the road for NLP regulation? AI companies that work closely with governments will lead the way, by necessity. Here at Primer, we build NLP systems for the US, UK, and Australian governments, helping with tasks as diverse as briefing report generation and disinformation detection. This front-row seat to the NLP revolution is humbling.
One thing is now clear. The machine learning tools that we are building today to process our language will transform our society. An intense research effort is underway to spot potential biases, harms, and mitigations. But every voice of hope and concern for this technology must be heard. Building NLP responsibly is our collective duty.

Read more: The Business Implications of Machines that Read and Write

How mapping bottom-up climate action can drive more strategic climate solution responses and help us adapt thoughtfully. 

At the recent COP26 climate summit, a Minister from the Pacific island country of Tuvalu announced that it would be seeking to retain legal status as a country even if its entire territory were to become submerged [Reuters]. He was standing thigh-deep in the ocean in an area that used to be dry land. His speech made it clear—the effects of climate change are here today.

When it comes to tackling the climate crisis, we typically think about solar power, electric vehicles, and carbon capture to mitigate future climate change. But Tuvalu’s story is the tip of the iceberg of climate adaptation — the messier, less-defined problem of how human civilization will respond to the changes that are here now and predicted to accelerate over the next 30 yrs — even if mitigation efforts are successful.

In order to help define this messy space, Primer recently partnered with Vibrant Data Labs, a social impact data science group, to make sense of this broader and more diverse climate landscape. Crunchbase and Candid provided data on over 12,000 companies and nonprofits funded in the past 5 years that are addressing climate-related topics. Primer’s natural language processing (NLP) engines mined these organizations’ descriptions to generate one of the first-ever, data-driven conceptual hierarchy of topics to better understand the shape of our current response, and its potential gaps. This unique perspective comes bottom-up from how the private and social sector organizations on the ground describe what they do — not by what is most spoken about in the news or social media.

Our analysis suggests that while new technologies are emerging to address climate mitigation, existing organizations that have historically tackled structural inequities (e.g, gender equity, migrant rights, homelessness) are uniquely poised to address climate adaptation challenges which permeate every aspect of civil society. Our sample showed these organizations are beginning to add a climate lens to their work on diverse social issues.

Defining the Climate Space

We created a hierarchy of interrelated topics based on the company descriptions. Using this hierarchy, we are able to surface the broad topics in climate work and also drill down into specifics. 

Examining the topics in this way revealed there are two major branches: one dealing with topics related to preventative technologies (Environment And Renewable Energy) and the other with topics addressing the human impact of change (Public And Social Benefits).  This computational technique led to a close split between mitigation and adaptation. It’s exciting that our method could organize these topics in a way that gets close to how a human would do the task.

The topics underneath our mitigation branch (Environment and Renewable Energy), are what one may expect: “Water”, “Nature and Conservation”, and “Energy, Storage, and Energy Waste”. Adaptation work is much more varied and therefore harder to define. Our analysis can help us paint a crisper image of this emerging landscape.

This topic hierarchy shows organizational distribution on climate change topics, with higher convergence at the top vs the bottom.

Climate Change as a Social Issue

The Intergovernmental Panel on Climate Change (IPCC) defines adaptation as “the process of adjustment to actual or expected climate and its effects” [IPCC]. Using this definition, we see some top level branches that are climate adaptation. As the earth warms and extreme weather becomes the new normal, Disaster Relief and Preparedness will be critical to serving areas affected.

A less obvious topic might be Community And Neighborhood Development. The subtopics within it seem like quite standard areas pertaining to social issues such as: Health, Affordable Housing, Human Rights, Government Advocacy, and Gender Equality. Looking deeper into the language of these organizations we can see how they are incorporating a climate lens to their work.  For example, here is the description of one of the organizations working in Gender Equality:

MADRE is an international women’s human rights organization founded 37 years ago.  We partner with grassroots women to create social change in contexts of war, disaster and injustice. To advance women’s human rights, MADRE supports grassroots women to meet their basic needs as a bridge to develop their advocacy and political participation….

…. Our actions to confront climate change today will decide the futures of our planet and of generations to come. You can join the women leading the way.    Climate change is a global threat, and poor, rural and Indigenous women are hardest hit. They are impacted first and worst by the food shortages, droughts, floods and diseases linked to this growing danger. But they are more than victims. They are sources of solutions, inventing innovative, locally-rooted responses.    Through our Women Climate Defenders initiative, MADRE’s grassroots partners enable their communities to adapt to climate change. They build clean water systems to guard against drought and seed banks to preserve future harvests.…

This is an example of how an organization that has been addressing women’s human rights for 37 years can contribute today to building climate resilience in the most vulnerable communities. It also highlights how climate adaptation requires addressing diverse, interdependent topics.

We can dive deeper into the Gender Equality data to understand the key topics that organizations in this field are working on today. A quick glance at this chart shows a wide range and diversity of topics in the climate adaptation cohort, from Human Rights to Infrastructure to Youth Organizing and Antidiscrimination.

The topics which co-occur most frequently with gender equality cover a range of socially minded topics which are not all tightly related to gender equality.

Let’s compare it to a topic from our climate mitigation set, Nature Conservation and Environment

The topics which co-occur most frequently with nature conservation and environment are very conceptually similar and mostly related to climate mitigation.

Organizations in this cohort work on Sustainability, Renewables, Water Conservation, Sustainable Agriculture, and Wildlife Preservation. It seems that most of these issues are more proximate to each other.

To further peel back the layers on current climate solutions, let’s take a deeper look at “crowding” or “spreading” of focus areas by organization. With NLP, we can approximately measure organizational “topic coherence” which tells us if a given organization optimizes on breadth or depth, and exactly how far apart the topics are within that cohort. We created a score from 0 to 1 that calculates how similar an organization’s topics are to each other—we call this the “organization focus score”. Organizations that focus on a narrow set of topics will have scores closer to 0. We can then extrapolate to the topic level to measure how narrowly focused the organizations in each topic are. When we plot this out from 0 to 1, we see topics relating to climate adaptation (Public and Social Benefit) are being addressed by organizations that are more broadly focused than the organizations addressing climate mitigation (Environment and Renewable Energy) topics.

The topic coherence score measures how closely related the topics a given organization works in are to each other. A score of 0 closer to 0 indicates the topics are very similar and a score closer to 1 means they are all very dissimilar. Climate adaptation topics (Public and Social Benefit) contain organizations with a more diverse set of focus areas than the climate mitigation topics.

Our analysis reveals that, while an organization working on mitigation will typically be working on a single, defined solution, organizations working on climate adaptation are fighting on multiple fronts.  


“In an interconnected world, it is exactly this messiness that funders need to embrace”, says Vibrant Data Labs’s Eric Berlow. “Traditional venture capital tends to fund focused, scalable solutions, with easy-to-measure outcomes, like renewable energy. But the climate crisis is an ‘all hands on deck problem’. Winning on one corner of the problem is an important piece; but if structural and systemic inequities in climate adaptation are not addressed, like the people of Tuvalu above, we all lose. We all feel the climate impacts of supply chain shocks, forced migration, civil unrest, and war. The most recent IPCC report suggests these trends will exacerbate over the next 30 years even if renewables and carbon capture solutions are successful. Climate funders will have to adopt a more holistic multi-pronged approach to rise to this challenge.”

Conclusion

As climate change becomes more and more a central part of our lives, understanding the landscape of solutions and providers gives us perspective on the magnitude of the space. We used NLP to analyze the work of over 12,000 companies to better understand where private and public organizations were focusing their efforts. In doing so, we highlighted the broad set of topics that are climate related and illustrated that many organizations working across the diverse social sector are now adding climate solutions to their efforts to enhance equity and resilience.

In a coming post, we will present our partner, Vibrant Data Labs’ story in which they take this analysis a step further to highlight the solution areas that are receiving the most funding.

Natural language processing (NLP) automates the work that would previously have required hundreds of researchers. These machines read and write – and they’re changing the future of business.

With NLP, we can now analyze information at machine speed, but with human-level precision. As reading and writing is automated through machine learning, companies are freed up to focus on the challenges that are uniquely human.

Read more here

Large organizations rely on hundreds, or even thousands, of vendors, partners, and supply chain businesses to provide key products and services. Primer’s advanced natural language processing (NLP) platform enables compliance teams to stay up-to-date on a large number of vendors and quickly surface business-critical information.

Read more here