To avoid undesirable content in the search indexes, webmasters can instruct spiders not to crawl certain files or directories through the standard robots.txt file in the root directory of the domain. Additionally, a page can be explicitly excluded from a search engine's database by using a meta tag specific to robots (usually ). When a search engine visits a site, the robots.txt located in the root directory is the first file crawled. The robots.txt file is then parsed and will instruct the robot as to which pages are not to be crawled. As a search engine crawler may keep a cached copy of this file, it may on occasion crawl pages a webmaster does not wish crawled. Pages typically prevented from being crawled include login specific pages such as shopping carts and user-specific content such as search results from internal searches. In March 2007, Google warned webmasters that they should prevent indexing of internal search results because those pages are considered search spam.
SEO often involves improving the quality of the content, ensuring that it is rich in relevant keywords and organizing it by using subheads, bullet points, and bold and italic characters. SEO also ensures that the site’s HTML is optimized such that a search engine can determine what is on the page and display it as a search result in relevant searches. These standards involve the use of metadata, including the title tag and meta description. Cross linking within the website is also important.
As mobile devices become an increasingly integral part of our lives, it’s vital that marketers understand how to effectively communicate on this unique and extremely personal channel. Mobile devices are kept in our pockets, sit next to our beds, and are checked constantly throughout the day. This makes marketing on mobile incredibly important but also very nuanced.
Off page SEO: This type of SEO focuses on all of the activity that takes place "off the page" when looking to optimize your website. "What activity not on my own website could affect my ranking?" You might ask. The answer is inbound links, also known as backlinks. The number of publishers that link to you, and the relative "authority" of those publishers, affect how highly you rank for the keywords you care about. By networking with other publishers, writing guest posts on these websites (and linking back to your website), and generating external attention, you can earn the backlinks you need to move your website up on all the right SERPs.