We’re No. 1! And also No. 119. And some numbers in between, too.
A seemingly endless stream of surveys has ranked San Diego’s business climate in the last few years and they’ve all drawn different conclusions.
In March, local boosters hailed Forbes’ declaration that San Diego is the best place to launch a start-up.
What’s the best city to start a business? Would you believe #SanDiego? Well, you would be right… http://t.co/2gZ2NONhn2
— Rep. Susan Davis (@RepSusanDavis) March 14, 2014
San Diego tops Forbes’ list of 10 Best Places to Launch a Startup. One reason: Lots of savvy social media users. http://t.co/KkwoEemDv6
— Downtown San Diego (@SDPartnership) March 13, 2014
.@Forbes says #SanDiego is the #1 place to start a business in 2014! http://t.co/SafS0IIxFe
— Kevin Faulconer (@kevin_faulconer) March 13, 2014
Unsurprisingly, a WalletHub report that ranked San Diego 119th out of 150 cities for new businesses just days earlier went largely ignored.
Two seemingly similar studies, one month, two strikingly different results.
So which rating is right? That depends entirely on your perspective.
These rankings and a host of others like them purport to measure what it’s like to do business here, a concept that draws disagreement even among business owners. To come up with these ratings, media outlets and think tanks pick and choose data sets, giving some data points more weight than others.
Then they release a list that says as much about those data points as the question that inspired the rankings.
For example, Forbes analyzed how small businesses fared and whether they had access to tools the publication deemed crucial to keeping those businesses afloat. Data points included the number of small businesses in high-growth industries, as well as the number of companies that accepted credit cards and had a web presence.
WalletHub, a social media website for personal finance, zeroed in on the conditions businesses face when moving into the largest US cities. Their rankings included a dive into a region’s cost of living, the overhead of starting a business and the level of local business competition.
The different metrics led to wildly contrasting pictures of San Diego’s business climate.
Here’s a look at various other ratings San Diego has received in recent history:
The scattershot results illustrate why rankings may be more useful to the politicos and economic-development gurus who cite them than to folks who want to understand the realities facing companies here.
“It doesn’t matter if it’s a media outlet or a respected think-tank ranking or a pseudo think-tank ranking. Rankings have problems,” said Yasuyuki Motoyama, a senior scholar for the Kansas City-based Ewing Marion Kauffman Foundation who has extensively studied such rankings. “It’s highly subjective and ultimately very questionable.”
Motoyama and a colleague even conducted a “create your own state rankings” experiment last year that helped prove his point. They chose several different variables to rank states on innovation and entrepreneurship and experimented with giving greater or lesser weight to each variable to see how states would stack up. Simply tweaking the relative importance of each data point left them with five different No. 1 states for innovation and entrepreneurship, and 16 that could be considered for the top five.
“There’s no such thing as an objective ranking,” Motoyama said. “It’s totally subjective.”
Economist Peter Fisher of the nonprofit Iowa Policy Project came to a similar conclusion in an assessment of a handful of national business rankings last May:
Each of these four rankings is constructed by taking widely disparate data points and adding or averaging them to construct an index number. The result is not a useful summary measure of business climate as claimed. It is at best meaningless, and at worst a state ranking manipulated to make the case for policy positions advocated by the organization sponsoring the index.
Not all such rankings are produced with political goals in mind.
What binds them, though, is an attempt to quantify a region’s business climate, a concept that’s defined more by perception than actual numbers.
“What they’re trying to measure is something that we don’t have a measure for: San Diego’s business friendliness,” said Erik Bruvold, an economic researcher who leads the National University System Institute for Policy Research in La Jolla.
Instead, those who aim to rank a region’s business competitiveness are left reaching for data that is available and provides a window – but not a full picture – into companies’ experiences in a particular place.
Bruvold and Motoyama both believe it’s impossible to measure a region’s business climate, though. Businesses and their needs vary too widely.
At least one more recent study tried to take differing opinions into account rather than just data.
Thumbtack, a San Francisco-based website that connects service-providing companies and customers, surveyed thousands of business owners across the country to produce its own small business friendliness survey.
San Diego fell to the bottom of their list, ranking as the fifth-least small business-friendly region among 82 metro areas. The ranking inspired few, if any, celebratory social media posts from local boosters.
Motoyama said he’s advised the website on past surveys and even cited it in a paper as an example of a potentially more responsible way to tap into various views about a given area’s business climate.
But he admitted the Thumbtack survey isn’t perfect either.
Business owners who use Thumbtack were prompted to take the questionnaire, meaning it wasn’t a random sample. Thumbtack also broadly refers to its study as the “Small Business Friendliness Survey” though three-quarters of business owners who took it work in the service industry, which would cover everything from a waste management company to one that provides IT help.
Thumbtack acknowledges manufacturing, retail and agriculture-tied companies were underrepresented in its survey but said respondents otherwise closely matched other demographics of the U.S. small business population.
Political pollster John Nienstedt, who also conducts regular business surveys for the San Diego Regional Chamber of Commerce, said a more comprehensive study would involve a random sample of businesses meant to represent the spectrum of companies and phone calls to ensure business owners who are less likely to respond online also voice their opinions.
The Thumbtack survey simply relied on web responses.
This is part of our quest digging into the difficulties – real or perceived – of doing business in San Diego. Check out the previous story in our series, San Diego Bucks California’s Trend in Biz Relocation, and the next, The Catch-22 of San Diego’s Tourism Economy.