AI Governance for Mid-Sized Companies

Less than most vendors selling governance software will tell you, but more than nothing. At 50 people, a one-page policy, a short list of approved tools, and a single owner is usually enough. At 100, add a lightweight approval process for new tools and a periodic review. At 500, you likely need defined roles, a review cadence tied to a real calendar, and enough documentation to show a client or auditor that governance is a practice, not an afterthought. The framework should scale with headcount and risk exposure, not with what a Fortune 500 template calls mature.

No — if anything, it's more urgent at mid-sized companies, which tend to have the AI exposure of a much larger company (employees using consumer AI tools daily) without the security team, legal department, or IT headcount to manage it. Enterprise companies often have this covered by function; a mid-sized company usually has one person quietly deciding.

Three things, roughly in order of how often they happen: client or company data ending up somewhere it shouldn't, typed into a free AI tool with no data-use agreement; a decision or deliverable built on AI output nobody verified; and inconsistent practice across teams that makes it hard to answer a client or partner who asks how you handle this. None of these require a dramatic breach to become a real problem.

Because AI adoption happened bottom-up and fast — one employee, one browser tab, one free account at a time — while governance requires someone to stop and build a policy, which is easy to deprioritize when nothing has gone wrong yet. RSM US's 2026 Middle Market Business Index Cybersecurity Special Report found that only 35% of middle-market executives reported having a formal AI governance framework in place, even as adoption keeps accelerating. Right now, a gap between AI use and AI oversight is the norm, not the exception.

Anything you wouldn't post publicly or hand to a competitor: unreleased financials, client contracts or pricing, personally identifiable employee or customer data, source code tied to a competitive advantage, and anything covered by an NDA. A useful gut check — if it would need a data processing agreement to leave the building for any other reason, it needs one before it goes into an AI tool too.

Generally, no — not because personal accounts are inherently unsafe, but because a personal account gives the company no visibility, no data-handling agreement, and no way to enforce a retention or deletion policy. If someone needs a tool, the fix is approving a business-tier account, not banning AI outright.

Only under a business or enterprise AI agreement that includes a data-processing addendum, and only with the customer's own contract in mind — some client agreements already restrict how their data can be used with third-party tools, AI included. Free-tier consumer AI tools should be treated as a hard no for customer data by default.

Mainly the contract. Free consumer tiers of tools like ChatGPT typically reserve the right to use conversations to train future models unless a user opts out, and offer no enforceable data-handling guarantees. Business and enterprise tiers come with a data processing agreement, opt-out of training by default, and often admin-level visibility and retention controls — which is what actually makes a tool safe to approve.

Not in the sense of one user's prompt showing up verbatim to a stranger — that's not how these tools work day to day. The real risk is subtler: data retained for model training, data stored longer than expected, or a vendor's own security incident exposing stored conversations. Treat "will this get used or stored somewhere I don't control" as the real question, not "will another user literally see it."

It matters, but a deletion promise in a free tool's terms of service isn't the same as an enforceable contractual commitment. If data handling genuinely matters for a given use case, that commitment needs to be in a business agreement your company can hold the vendor to — not a line in a consumer terms-of-service page that can change at any time.

Whichever ones people are actually using or asking to use — the point of an approved list is to cover real behavior, not an idealized one. Start by asking teams what they've already adopted, then evaluate each for a business-tier option, a real data-handling agreement, and fit with an existing workflow, before locking in a short approved list.

Find out what's actually in use before writing any policy; a policy built without that step just gets worked around. Then convert the tools that are genuinely useful into approved, business-tier versions rather than banning them outright — most shadow AI isn't malicious, it's someone solving a real problem with the only tool they had access to.

Shadow AI is any AI tool being used for company work without IT or leadership's knowledge or approval — a personal ChatGPT account, a browser extension, an AI feature quietly turned on inside another SaaS product. You typically find out one of two ways: someone asks about it directly, which is the better outcome, or it surfaces after something has already gone wrong.

Blocking outright almost never works and mostly pushes usage further out of sight, onto personal devices and personal accounts where there's even less visibility. An approved list, reviewed periodically, gets far better real-world compliance than a ban.

Something short enough that people will actually use it: what the tool does, whether a business-tier plan with a data agreement exists, who's requesting it and why, and a yes or no from whoever owns the policy. A one-page request form beats a committee review that takes six weeks — by then, people have usually started using the tool anyway.

Yes, and it's usually the first thing worth fixing. Free tiers are the versions most likely to use conversation data for model training, least likely to offer a data processing agreement, and hardest to get visibility into. If a tool is genuinely useful, the fix is usually upgrading to its business tier, not banning it.

The person who used it and the person who approved it for use — not the AI tool, and not "the algorithm." That's true whether the mistake is a factual error in a client deliverable or a flawed recommendation in an internal decision. Treating AI output as a draft a human is accountable for, every time, is the single most protective habit a company can build.

For anything client-facing, anything with numbers in it, or anything that could carry legal or reputational weight — yes, always. For low-stakes internal drafts, a lighter review is reasonable. The line isn't "did AI touch this," it's "what happens if this specific piece is wrong."

The same thing that happens with any bad decision based on bad information — it's the company's decision to own and correct, not a defense that shifts blame to a tool. This is exactly why review and verification matter more than which AI tool was used.

Increasingly yes for anything client-facing, and it's worth doing even when it isn't strictly required — most clients care less that AI was involved and more whether a human reviewed and stands behind the result. Some client contracts are starting to require disclosure explicitly, so it's worth checking existing agreements rather than assuming silence means it isn't covered.

No — using AI doesn't reduce or transfer liability for what the company delivers under a client contract. If anything, it raises the bar: "we didn't catch it" is a weaker position when the tool that produced the error was something the company chose to adopt.

Five things, at minimum: which tools are approved, what data can and can't be used with them, who owns AI decisions and enforcement, how AI-assisted work gets reviewed before it ships, and what happens when something goes wrong. Everything else — steering committees, formal risk frameworks, detailed enforcement mechanics — can be added later as the company's AI use matures.

At least twice a year, and immediately after any major new tool adoption, a near-miss, or a relevant regulatory change. AI tools and their data practices change faster than most companies update any other internal policy — an 18-month-old AI policy is close to useless.

One named person, not a committee — usually whoever already owns IT, security, or operations, with input from legal on the data-handling language. The specific title matters less than having someone whose job it clearly is to keep the policy current and answer "can I use this tool for this" questions as they come up.

For most mid-sized companies, not the full framework. NIST's AI RMF — organized around four functions: Govern, Map, Measure, and Manage — is a useful mental model to borrow from, especially the idea of naming clear ownership and roles, but adopting the full framework is usually more structure than a company under a few hundred employees needs. Use it as a checklist to sanity-check a simpler, in-house policy rather than a document to formally adopt.

Find out what AI tools are already in use, write a one-page policy covering the five basics above, and name one owner — in that order. That's a governance program a company can actually run, and it can grow into something more formal later. A company with no policy today doesn't need a mature framework; it needs a starting point.