{"version":"1.0","provider_name":"Hosteko Blog","provider_url":"https:\/\/hosteko.com\/blog","author_name":"Fitri Ana","author_url":"https:\/\/hosteko.com\/blog\/author\/fitriana","title":"Cara Block Bot Search Engine Melalui Robots.txt dengan Benar dan Aman - Hosteko Blog","type":"rich","width":600,"height":338,"html":"<blockquote class=\"wp-embedded-content\" data-secret=\"tSu0p2Z7Ci\"><a href=\"https:\/\/hosteko.com\/blog\/cara-block-bot-search-engine-melalui-robots-txt\">Cara Block Bot Search Engine Melalui Robots.txt dengan Benar dan Aman<\/a><\/blockquote><iframe sandbox=\"allow-scripts\" security=\"restricted\" src=\"https:\/\/hosteko.com\/blog\/cara-block-bot-search-engine-melalui-robots-txt\/embed#?secret=tSu0p2Z7Ci\" width=\"600\" height=\"338\" title=\"&#8220;Cara Block Bot Search Engine Melalui Robots.txt dengan Benar dan Aman&#8221; &#8212; Hosteko Blog\" data-secret=\"tSu0p2Z7Ci\" frameborder=\"0\" marginwidth=\"0\" marginheight=\"0\" scrolling=\"no\" class=\"wp-embedded-content\"><\/iframe><script>\n\/*! This file is auto-generated *\/\n!function(d,l){\"use strict\";l.querySelector&&d.addEventListener&&\"undefined\"!=typeof URL&&(d.wp=d.wp||{},d.wp.receiveEmbedMessage||(d.wp.receiveEmbedMessage=function(e){var t=e.data;if((t||t.secret||t.message||t.value)&&!\/[^a-zA-Z0-9]\/.test(t.secret)){for(var s,r,n,a=l.querySelectorAll('iframe[data-secret=\"'+t.secret+'\"]'),o=l.querySelectorAll('blockquote[data-secret=\"'+t.secret+'\"]'),c=new RegExp(\"^https?:$\",\"i\"),i=0;i<o.length;i++)o[i].style.display=\"none\";for(i=0;i<a.length;i++)s=a[i],e.source===s.contentWindow&&(s.removeAttribute(\"style\"),\"height\"===t.message?(1e3<(r=parseInt(t.value,10))?r=1e3:~~r<200&&(r=200),s.height=r):\"link\"===t.message&&(r=new URL(s.getAttribute(\"src\")),n=new URL(t.value),c.test(n.protocol))&&n.host===r.host&&l.activeElement===s&&(d.top.location.href=t.value))}},d.addEventListener(\"message\",d.wp.receiveEmbedMessage,!1),l.addEventListener(\"DOMContentLoaded\",function(){for(var e,t,s=l.querySelectorAll(\"iframe.wp-embedded-content\"),r=0;r<s.length;r++)(t=(e=s[r]).getAttribute(\"data-secret\"))||(t=Math.random().toString(36).substring(2,12),e.src+=\"#?secret=\"+t,e.setAttribute(\"data-secret\",t)),e.contentWindow.postMessage({message:\"ready\",secret:t},\"*\")},!1)))}(window,document);\n<\/script>\n","thumbnail_url":"https:\/\/hosteko.com\/htk-blog\/wp-content\/uploads\/2026\/01\/Tambahkan-judul1.png","thumbnail_width":1366,"thumbnail_height":768,"description":"Robots.txt adalah file penting dalam pengelolaan website yang berfungsi untuk memberikan instruksi kepada bot atau crawler mesin pencari mengenai halaman atau direktori mana yang boleh dan tidak boleh diakses. Dengan pengaturan robots.txt yang tepat, pemilik website dapat mengontrol proses crawling, melindungi area tertentu, serta mengoptimalkan performa SEO. Artikel ini akan membahas secara lengkap dan detail [&hellip;]"}