A Short History of SEO for Digital Marketing Beginners

Most people are at least broadly familiar with the concept of search engines. You type the word or phrase for which you need information, and the tool goes and fetches the best results, right? This is not a new concept. But comparatively few people have any idea what’s going on under the hood or how the search engine algorithms actually work.

Colibri Digital Marketing is a top San Francisco digital marketing agency. We’re right next door to Silicon Valley, and we get more than a few questions about how search engines function, and in order to give our answers some context, we always end up explaining the history of search engine optimization (SEO).

If you’re curious about how search engines came to be, read this post explaining the history of SEO. You’ll learn where it all began, and why. You’ll learn the basic principles of SEO, and you’ll get a feel for the 2017 SEO landscape from the perspective of a top San Francisco digital marketing agency. Happy reading!

What Is SEO?

So, what exactly is a search engine and what does it aim to do?

Developing a Search Engine

That kind of archive-and-retrieval system would take a number of different forms in the decades to come, but what we today think of as a full search engine got started in the early 90s.

In February of 1993, a group of Stanford students (you’ll find that “a group of Stanford students” is something of a running theme) created the foundation for Excite, one of the first search engines. Their system, Architext, sorted information by keywords extracted from the content. By December of that year, a number of other search engines (JumpStation, RBSE spider, and World Wide Web Worm, among others) were in operation which used “bots” (crawlers, spiders, trawlers, etc.) to evaluate and catalog webpages to improve search results for users.

By mid 1994, Alta Vista, Yahoo, Lycos, and Infoseek were all in play, and the notion of a “search engine” was gradually permeating the public consciousness. Things really took off, however, in 1996, when Stanford students Sergey Brin and Larry Page began building what would develop into the biggest, most recognizable search engine in the world: BackRub.

What? You thought I was going to say “Google,” didn’t you? Well, you’re not entirely wrong. BackRub would evolve into Google (the domain was registered in 1997) with the advent of PageRank.


SPAM and Misuse

Today, those shady tactics fall under the broad category of “black hat SEO’ but back then, they weren’t being restrained or punished. It wasn’t good for Google, it wasn’t good for users, and it wasn’t really any good for the sites themselves.

Something had to change.


Machine Learning and Natural Language Processing

Put yourself in the mind of a basic algorithm for a second. You’ve been told to cull pages that seem to overuse phrases that would lend themselves to search. Can you tell the difference between these two passages?

“The Shiba Inu is a small, agile dog that copes very well with mountainous terrain. The Shiba Inu was originally bred for hunting. It looks similar to and is often mistaken for other Japanese dog breeds like the Akita Inu or Hokkaido, but the Shiba Inu is a different breed with a distinct blood line, temperament and smaller size than other Japanese dog breeds.”


“Shiba Inu dog is Japanese hunting dog breed Hokkaido is small agile dog breed and good dog for rough terrain like akita husky Hokkaido shiba inu Japanese dog breeds.”

Since you, the simple algorithm, lack any real understanding of the meaning of the words, you have no context for the keywords you see, so you cull pages indiscriminately.

It’s not a great system.

What Google lacked in 2003 was a way to algorithmically quantify meaning and context.

Panda and Penguin

What they had in common was the ability to more accurately quantify the context of their subjects.

Penguin, for instance, was able to discern the relevance of a backlink from the semantic content of the page. An article on dogs, say, which linked to a page about the Shiba Inu would be a more valid backlink than a directory listing of unrelated webpages.

From then onward, Google would continue refining their algorithms as their tech evolved, with the ultimate goal of creating a simple algorithm that would parse text with the same kind of understanding as a human user, thereby to present that user with the best possible results.

SEO in 2017

We’re not quite there yet, but we’re as close as we’ve ever come to real natural language processing by the various algorithms and systems which ultimately feed the SERPs. More than ever before, strong, relevant content is being promoted while thin, spammy content is being suppressed.

So, what does this mean for you?

(For more SEO tips, check out our post on off-page SEO.)

So, there you have it. SEO in a nutshell from its inception right through to the present day. If you’ve got any comments or questions, don’t hesitate to hit us up on Facebook or Twitter or reach out with the big friendly button below to sign up for a complimentary digital marketing strategy session, and find out how we can help you grow your digital presence.

Colibri Digital Marketing

We’re the digital marketing agency San Francisco trusts to focus on the triple bottom line of people, planet, and profit. Based in the Bay Area, close to Silicon Valley, we’re the team with the sneak-peek into the future of digital marketing. If you’re ready to work with San Francisco’s first and only full-service B Corp-Certified digital marketing agency, drop us a line or click to schedule a free digital marketing strategy session!

Originally published at colibridigitalmarketing.com on August 31, 2017.

I work to make the web a more beautiful, accessible, and functional place. I use dreams as a form of planning. And I play because it’s fun.