Testimony of Jordan Shapiro: Oversight: Examining the Risks Posed by Artificial Intelligence (T2026-2573)
Committee of the Whole
City Hall, New York, NY 10007
Testimony of Jordan Shapiro: Oversight: Examining the Risks Posed by Artificial Intelligence (T2026-2573)
My name is Jordan Shapiro. I am the Founder of JD Digitalization, a technology, operations, and responsible tech consultancy. Thank you, Speaker Menin, Chair De La Rosa, and members of the Council, for the opportunity to testify.
First, a question about trust: When AI companies tell us they have safe, private tools, we ask for proof. Yet, when the industry's loudest voices say they have built something that could end humanity, why do we take that on faith?1 Sensational claims travel fastest, in our feeds and in our headlines.2a,2b,2c
What we can do is look closely at the evidence. The so-called rogue agents in the news these last couple of months have turned out to be far more ordinary than the sensationalist claims would lead us to believe.3a,3b,3c,3d Two found private API keys publicly listed by humans.4a,4b One found a security bug.5a,5b And the Hugging Face breach began with a test that accidentally included impossible tasks. The models kept going until they broke out and went looking for the answers online.6a,6b,6c,6d
I'm not saying there is no risk. We've already seen extremely capable models be released and then taken off the shelf,7 and we have no idea what, behind trade secrets, these companies have built. It could be dangerous. It also could be fuel for relevance around Initial Public Offerings.8a,8b They aren't obligated to tell us.
The reality is we have very little power to change the incentives these companies have to block bills that limit their power, to seek profit, and push sensationalism over the public interest.9
Many of the bills on today's agenda are aimed at these companies, and I understand why. Federal regulators have sued these companies, even won.10 States have passed privacy and consumer protection laws.11 Corporate power has only grown. Regulating companies matters. But it may not be where this body has the most power or impact.
The city's own report calls last year's 56% jump in its algorithmic tools "the largest to date,"12a,12b and the rules for its generative AI tools are still mostly suggestions.13a,13b From my position as a full stack tech policy expert, that is one of the biggest risks to New Yorkers. City staff already use ChatGPT, Microsoft Copilot, Google Gemini, and Anthropic's Claude in at least eight agencies and mayoral offices, according to the city's own report of its algorithmic tools for 2025.14a,14b,14c,14d,14e,14f,14g,14h Six of them say identifying information may be going into those tools.15a,15b,15c,15d,15e These are the AI risks this Council can act on.
Yet the city's own guidance warns that "GenAI systems not designed for enterprise use often lack essential protections."16 And yet, these are the tools some of our agencies are using without official standards, which the new Office of Algorithmic Accountability will be writing.17a,17b Even the city's guidance asks, "What are the acceptable levels of performance?" without answering.18 Today, using these tools responsibly depends on each city employee remembering the guidance. Responsible digital systems don't depend on memory. They use least privilege, meaning every person and tool gets only the access its job requires, with clear limits on what can and cannot be done, built into the system itself.
City Comptroller Mark Levine put it plainly on September 24: "Our government is built on a rickety tech foundation that includes dozens of mainframes from the 1980s. We largely missed the cloud revolution, the mobile revolution, and the big data revolution, and are now dragging our feet on the machine-learning revolution."19
But it doesn't have to be like this.
Roll out AI to city agencies properly, with a tool the city has chosen and tested, clear guidelines for how to use it, and training and checks for the people who do. Council Member De La Rosa said it best: "New York City is already at the forefront of emerging technology, but it should also set an example for the responsible application of AI."20
Organizations and individuals that are getting the most out of AI aren't using the out-of-the-box instance. We are all aware that we don't have the reins or the data on how well AI companies took into account anti-bias or fairness or non-exploitation in their training and by using the free, commercially available tools, we are also saddled with that unknown.
Instead these organizations write their own rules that govern the tool, whether it's a simple brand kit or privacy guidance. The AI I work with everyday never asks for personal data or adds it to chat or logs, it opens a source before describing it, and does not tokenmaxx (using as many tokens as possible). These rules come from the governance layer I designed for my own system. I work with non-profits who set a similar system up, and in my client discussions I have learned of other companies taking the same approach, without compromising their data or intellectual property.21a,21b,21c The city can do the same.
I ask the Council to:
Pick a model and negotiate the terms. Whether it's a local, offline model, or a commercial model, the city, ideally, or agencies, should pick a model or system under one contract negotiated for every agency that keeps city data private. Then program it by adding a governance layer and universal rules that protect New Yorkers and follow the city's values.
Define what it means when AI "works," and test against it. Local Law 193 makes the Office of Algorithmic Accountability responsible for the standards every agency must meet when it uses AI that affects the public, including "regular monitoring and evaluation."17a,17b Hold the office to it: set a performance bar for each use, test the tools staff already use against it, and add those results to the public list of assessed AI systems that Local Law 195 requires by March 31, 2027.22
Train people to use it. AI is a powerful tool, it can do a lot of things, and people have to be trained on the city's values and acceptable use. The city is hiring technologists for its new PIT crews.23 Task them with training coordinators in each agency, and have those coordinators train their colleagues, the way the city's Open Data Coordinators already work.
I started with a question about trust. No single bill or package of bills from this chamber can change the incentives these companies answer to and make them more trustworthy. What you can do is build trust in how New York City uses their tools: pick a model that keeps New Yorkers' data private, define what it means for that model to "work" and test it, and train people to use it. That is readiness. A city that is ready for the everyday use of AI is much better prepared to deal with the risks that may come next.
Bibliography
::: {.bib} Anthropic. "Anthropic Confidentially Submits Draft S-1 to the SEC." June 1, 2026. https://www.anthropic.com/news/confidential-draft-s1-sec.
Anthropic. Commercial Terms of Service. Effective June 17, 2025. https://www.anthropic.com/legal/commercial-terms.
Associated Press. "Google Loses Massive Antitrust Case over Its Search Dominance." OPB, August 5, 2024. https://opb.org/article/2024/08/05/google-loses-massive-antitrust-case-over-its-search-dominance.
Associated Press. "OpenAI's Breach of Australian Health Department Website Prompts Rebuke." NPR, September 24, 2026. https://www.npr.org/2026/09/24/g-s1-144835/openai-breach-australia.
Bellan, Rebecca. "a16z-Backed Super PAC Is Targeting Alex Bores, Sponsor of New York's AI Safety Bill." TechCrunch, November 17, 2025. https://techcrunch.com/2025/11/17/a16z-backed-super-pac-is-targeting-alex-bores-sponsor-of-new-yorks-ai-safety-bill-he-says-bring-it-on.
Bhaimiya, Sawdah. "Anthropic Warns Investors of AI's 'Existential Risk to Humanity' in IPO Prospectus, Reports Say." CNBC, September 28, 2026. https://www.cnbc.com/2026/09/29/anthropic-warns-ai-existential-risks-ipo-filing-reuters.html.
Brady, William J., Julian A. Wills, John T. Jost, Joshua A. Tucker, and Jay J. Van Bavel. "Emotion Shapes the Diffusion of Moralized Content in Social Networks." Proceedings of the National Academy of Sciences 114, no. 28 (2017): 7313–18. https://doi.org/10.1073/pnas.1618923114.
Cable, Jack, et al. "Early Rogue AI Agent Activity and Attempts to Hack Found on urlquery.net." Transluce, September 23, 2026. https://transluce.org/agent-activity.
Capoot, Ashley. "OpenAI Cyber Models Broke Out of Training Environment to Hack Hugging Face." CNBC, July 22, 2026. https://www.cnbc.com/2026/07/22/open-ai-cyber-models-hack-hugging-face.html.
Hoblitzell, Andrew. "Anthropic Releases and Temporarily Suspends Claude Fable 5." InfoQ, June 15, 2026. https://www.infoq.com/news/2026/06/claude-5-release/.
International Association of Privacy Professionals. "US State Privacy Legislation Tracker." https://iapp.org/resources/article/us-state-privacy-legislation-tracker/.
Lanz, Jose Antonio. "OpenAI Halts Model Training as Rogue Agents Target US Government Sites." Decrypt, September 28, 2026. https://decrypt.co/379508/openai-ai-agents-target-us-government-sites.
Levine, Mark. "NYC Is Not Ready for the AI That's Already Here." Mark Levine's Substack, September 24, 2026. https://marklevinenyc.substack.com/p/nyc-is-not-ready-for-the-ai-thats.
New York City Council. "New York City Council to Hold Major Hearing on Artificial Intelligence Risks, Examine Need for Stronger Protections for New Yorkers." Press release, September 16, 2026. https://council.nyc.gov/press/2026/09/16/3240/.
New York City Local Law No. 193 of 2025 (Int. No. 926-A). https://intro.nyc/0926-2024.
New York City Local Law No. 195 of 2025 (Int. No. 1024-A). https://legistar.council.nyc.gov/View.ashx?M=F&ID=15062374&GUID=A4A39C1B-F7C5-4410-939C-FB0D4BACDCF7.
New York City Office of Technology and Innovation. LL35: Algorithmic Tools, CY 2025. New York: City of New York, March 27, 2026. https://www.nyc.gov/assets/oti/downloads/pdf/reports/LL35%20Report%202025%20-%20Final%20-%202026-03-27.pdf.
New York City Office of Technology and Innovation. Use Guidance: Generative Artificial Intelligence. Version 2.0. New York: City of New York, December 30, 2025. https://www.nyc.gov/assets/oti/downloads/pdf/New-York-City-Generative-AI-Use-Guidance.pdf.
New York City Office of the Mayor. "Mayor Mamdani Launches 'Public Interest Technology (PIT) Crew' to Rapidly Build Digital Solutions to Public Problems." Press release, July 13, 2026. https://www.nyc.gov/mayors-office/news/2026/07/mayor-mamdani-launches--public-interest-technology--pit--crew--t.html.
OpenAI. "Business Data Privacy, Security, and Compliance." https://openai.com/business-data/.
OpenAI. OpenAI – Hugging Face Incident Technical Report. 2026. https://cdn.openai.com/pdf/67869394-cb91-4c12-888c-5cbd85c7814c/OpenAI-Hugging-Face%20Incident-Technical-Report.pdf.
Park, Kate. "OpenAI Apologizes to Australia after Its AI Agents Breached Government Sites." TechCrunch, September 29, 2026. https://techcrunch.com/2026/09/29/openai-apologizes-to-australia-after-its-ai-agents-breached-government-sites/.
Robertson, Claire E., Nicolas Pröllochs, Kaoru Schwarzenegger, Philip Pärnamets, Jay J. Van Bavel, and Stefan Feuerriegel. "Negativity Drives Online News Consumption." Nature Human Behaviour 7, no. 5 (2023): 812–22. https://doi.org/10.1038/s41562-023-01538-4.
Vosoughi, Soroush, Deb Roy, and Sinan Aral. "The Spread of True and False News Online." Science 359, no. 6380 (2018). https://www.media.mit.edu/publications/the-spread-of-true-and-false-news-online/. :::