To avoid undesirable content in the search indexes, webmasters can instruct spiders not to crawl certain files or directories through the standard robots.txt file in the root directory of the domain. Additionally, a page can be explicitly excluded from a search engine's database by using a meta tag specific to robots (usually  ). When a search engine visits a site, the robots.txt located in the root directory is the first file crawled. The robots.txt file is then parsed and will instruct the robot as to which pages are not to be crawled. As a search engine crawler may keep a cached copy of this file, it may on occasion crawl pages a webmaster does not wish crawled. Pages typically prevented from being crawled include login specific pages such as shopping carts and user-specific content such as search results from internal searches. In March 2007, Google warned webmasters that they should prevent indexing of internal search results because those pages are considered search spam.[47]

While most search engine companies try to keep their processes a secret, their criteria for high spots on SERPs isn't a complete mystery. Search engines are successful only if they provide a user links to the best Web sites related to the user's search terms. If your site is the best skydiving resource on the Web, it benefits search engines to list the site high up on their SERPs. You just have to find a way to show search engines that your site belongs at the top of the heap. That's where search engine optimization (SEO) comes in -- it's a collection of techniques a webmaster can use to improve his or her site's SERP position.
And why not? Human beings have always enthralled themselves into one pursuit after another, all with a means to an end of improving our lives. Clearly, the conveniences afforded by the internet are quite literally earth-shattering to say the least. Three decades ago, few could have ever imagined the present state of our on-demand-everything society, with the ability to instantly communicate and conduct business in real-time, at a pace that often seems dizzying at the best of times.
Google's search engine marketing is one of the western world's marketing leaders, while its search engine marketing is its biggest source of profit.[17] Google's search engine providers are clearly ahead of the Yahoo and Bing network. The display of unknown search results is free, while advertisers are willing to pay for each click of the ad in the sponsored search results.
Connecting the dots between marketing and sales is hugely important -- according to Aberdeen Group, companies with strong sales and marketing alignment achieve a 20% annual growth rate, compared to a 4% decline in revenue for companies with poor alignment. If you can improve your customer's' journey through the buying cycle by using digital technologies, then it's likely to reflect positively on your business's bottom line.
The new digital era has enabled brands to selectively target their customers that may potentially be interested in their brand or based on previous browsing interests. Businesses can now use social media to select the age range, location, gender and interests of whom they would like their targeted post to be seen by. Furthermore, based on a customer's recent search history they can be ‘followed’ on the internet so they see advertisements from similar brands, products and services,[40] This allows businesses to target the specific customers that they know and feel will most benefit from their product or service, something that had limited capabilities up until the digital era.
Inbound marketing refers to a marketing methodology wherein you attract, engage, and delight customers at every stage of the buyer's journey. You can use every digital marketing tactic listed above, throughout an inbound marketing strategy, to create a customer experience that works with the customer, not against them. Here are some classic examples of inbound marketing versus traditional marketing: 
×