Last week, I shared our work at Esri adding generative AI to ArcGIS. Over the past year, I’ve been leading a new ArcGIS Hub AI Assistant in close collaboration with the city of Washington, DC to support easy access and understanding of open data about the city. This work builds from my talk at the National Academies on the future of GeoAI and the potential for improved access to information.
The future is exciting, and these AI assistants will be progressively released throughout 2024. They are just a few examples of the many generative AI related projects across Esri.
My colleague, Adam Pfister, and I presented at the Esri FedGIS 2024 conference. You can watch the video recording here or read the transcript below which may be easier to read and understand what we’re doing.
ArcGIS and Generative AI Assistants
Over the past several years, Esri has been adding machine learning and GeoAI to ArcGIS to provide you with advanced spatial analysis capabilities.
Last year, there was a revolutionary shift with the development of Large Language Models, or LLMs, used in Generative AI. These advancements have opened new avenues for interactive, accessible experiences. You might have heard of Microsoft’s CoPilot and OpenAI’s ChatGPT. These capabilities change how people work by providing creative reasoning and natural language interfaces.
Across Esri, we’ve been exploring new capabilities that use generative AI to automate and accelerate workflows.
It starts by following the WebGIS pattern. Publish your data as services, including high-quality metadata. Use these data services as layers in a map, and use in Applications.
We are developing new “AI Assistants” that extend these Apps which use Agents with specialized training, prompt engineering, and LLM to provide particular skills. These use our Tools to access the live data and services – which is important to ensure that answers are accurate and trustworthy.
We envision generative AI being available across ArcGIS. I’m going to show you a few examples of our research developing AI Assistants into ArcGIS Hub, Survey123, and integrates ArcGIS System.
Open Data DC
The city of Washington, DC uses hub to share their data, documents, and apps through a focused catalog that supports constituents, partners, and developers to use in their own decision making. The data can be explored, downloaded, access via APIs, and integrated into configurable apps.
Our research indicated that generative AI could help people more quickly & easily get answers to common questions. We are developing a new Hub AI Assistant that will be integrated into ArcGIS Hub and configurable for your organizations’ sites.
We connected the Hub AI Assistant to DC’s Open Data site so that all responses are using their authoritative, up-to-date data and information. When I moved into the city, my first question was “when is trash day”
Now, people can ask this common question to discover relevant information to explore. They can then find the days specific to their address – let’s see at “check at 800 east capitol st ne”
With generative AI it’s important to ensure the source data are accurate, so the AI Assistant provides links to the data source as well as a map and the geocoded location so the person can verify the information is correct.
You can also ask broader questions like “How many trees are in the city?” The DC Urban Forestry team publishes datasets with each tree, species, size, and health. You can then ask it to create a map of trees nearby our location. And because the AI Assistant is using the live services – when the forestry team plants a new tree, the next time someone asks the question they will see the newest trees.
The Hub AI Assistant answers general questions using the documents, and apps in the city’s Hub. I’m wondering “who can help me plant a tree”. There is a simple response and link to the hub page with more information. This helps people understand policies and procedures alongside data exploration.
This is an exciting future and you’ve probably already done the hard work by publishing your data as services and adding terrific metadata that the AI Assistants will include in their reasoning. It’s important to realize that the use of your data in generative AI is happening already and accelerating. We are mitigating the risks by limiting inputs to your curated data and configurable guardrails. There is a lot of work to do here, and we’re being careful about how to respond to conversations outside the scope of the assistant.
For example, someone may say “my house is on fire”. The AI Assistant limits its reasoning and information to the curated data and clearly states when a question is outside of its scope. In this case – call emergency services using 911.
The AI Assistant also helps ensure everyone has easy access to the data using their native language. Parents want to know where their child should attend school, “¿Cuáles son las escuelas cerca del círculo DuPont?” The AI Assistant translates questions to query the data and find the best answer. Then it responds in the language the person asked.
Survey123 Assistant
Survey123 is a powerful capability for creating dynamic forms to gather information. However, researching and formulating questions and answers can take significant time. Then we need to configure the form with all the details.
The upcoming Survey123 Assistant accelerates your work. We can request it to “create a survey for reporting damage assessments due to flooding”. The Survey123 Assistant creates a draft survey based on training using domain knowledge relevant to the request and prompt engineering for best practices. Within a few seconds we have a suggested set of questions and answers.
We can modify this to “add a map for marking the location”. The assistant will consider my request, update the questions and answers, then provide a complete draft for me to review.
When we’re satisfied with this draft the Survey123 Assistant can Generate the form automatically. Now we can continue working in the form editor to make changes, share with colleagues and publish when we’re confident.
These are just two examples of our research and development efforts integrating generative AI into new AI Assistants.
We are testing these AI Assistants for ArcGIS with different users and organizations to progressively improve their quality, utility, and accessibility.