{"id":29425,"date":"2026-01-17T08:11:24","date_gmt":"2026-01-17T08:11:24","guid":{"rendered":"https:\/\/hosteko.com\/blog\/?p=29425"},"modified":"2026-01-17T08:11:24","modified_gmt":"2026-01-17T08:11:24","slug":"mengenal-robots-txt","status":"publish","type":"post","link":"https:\/\/hosteko.com\/blog\/mengenal-robots-txt","title":{"rendered":"Apa Itu Robots.txt? Panduan Lengkap Fungsi, Struktur, dan Cara Membuatnya"},"content":{"rendered":"

Robots.txt adalah salah satu file paling penting dalam pengelolaan website dan SEO, meskipun ukurannya sangat kecil. File ini berfungsi sebagai \u201cpeta aturan\u201d yang memberi tahu mesin pencari bagaimana mereka boleh menjelajahi website kamu.<\/p>\n

Pengaturan robots.txt yang tepat akan membantu website lebih cepat terindeks, lebih aman, dan lebih optimal di mesin pencari. Artikel ini membahas secara sangat lengkap dan terstruktur per poin, mulai dari pengertian, fungsi, cara kerja, struktur, hingga praktik terbaik penggunaannya.<\/p>\n

Pengertian Robots.txt<\/strong><\/h2>\n

Robots.txt adalah file teks yang diletakkan di direktori utama (root) website yang digunakan untuk mengatur akses bot mesin pencari terhadap halaman dan folder tertentu. File ini menggunakan standar Robots Exclusion Protocol (REP) yang dipatuhi oleh crawler seperti Googlebot, Bingbot, dan YandexBot. Lokasi file: https:\/\/namadomain.com\/robots.txt<\/code> ,Tanpa file ini, bot tetap bisa meng-crawl website, tetapi tanpa panduan yang jelas tentang prioritas dan batasan akses.<\/p>\n

Cara Kerja Robots.txt<\/strong><\/h2>\n
\n
\n
\n
\n
\n
\n
\n
\n

Saat bot mesin pencari seperti Googlebot, Bingbot, atau YandexBot mengunjungi sebuah website, langkah pertama yang mereka lakukan adalah mencari file robots.txt di root domain. File ini berfungsi sebagai panduan awal yang memberi tahu bot bagaimana mereka harus menjelajahi website tersebut.<\/p>\n

Setelah menemukan robots.txt, bot akan membaca setiap aturan di dalamnya untuk memahami bagian mana dari website yang boleh di-crawl, bagian mana yang harus dihindari, serta file atau folder mana yang diizinkan untuk diakses. Berdasarkan aturan ini, bot akan menyusun rencana penjelajahan (crawl plan) sehingga waktu dan sumber daya mereka digunakan secara efisien untuk halaman yang benar-benar penting.<\/p>\n

Jika sebuah halaman atau folder diblokir di dalam robots.txt menggunakan perintah Disallow<\/code>, bot tidak akan melakukan proses crawling terhadap halaman tersebut, sehingga kontennya tidak dianalisis lebih lanjut. Namun, perlu dipahami bahwa pemblokiran ini tidak selalu berarti halaman tersebut pasti tidak muncul di hasil pencarian.<\/p>\n

Jika halaman tersebut memiliki tautan dari website lain, mesin pencari masih bisa menampilkan URL-nya di hasil pencarian tanpa deskripsi lengkap. Oleh karena itu, robots.txt sebaiknya digunakan untuk mengatur proses crawling, bukan sebagai satu-satunya metode untuk menyembunyikan halaman, dan sering dikombinasikan dengan tag noindex<\/code> agar halaman benar-benar tidak ditampilkan di hasil pencarian.<\/p>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/article>\n<\/div>\n

Fungsi Robots.txt Secara Detail<\/strong><\/h2>\n

1. Mengontrol Akses Bot Mesin Pencari<\/strong><\/p>\n

Robots.txt menentukan halaman penting yang harus diprioritaskan dan halaman tidak penting yang sebaiknya dihindari.<\/p>\n

2. Menghemat Crawl Budget<\/strong><\/p>\n

Mesin pencari memiliki batas kunjungan ke website. Robots.txt membantu agar crawl budget digunakan untuk halaman penting seperti artikel dan produk.<\/p>\n

3. Melindungi Halaman Sensitif<\/strong><\/p>\n

Contohnya:<\/p>\n

    \n
  • \/wp-admin\/<\/code><\/li>\n
  • \/login\/<\/code><\/li>\n
  • \/cart\/<\/code><\/li>\n
  • \/checkout\/<\/code><\/li>\n<\/ul>\n

    4. Menghindari Konten Duplikat<\/strong><\/p>\n

    Mengurangi crawl pada:<\/p>\n

      \n
    • Tag<\/li>\n
    • Filter<\/li>\n
    • Parameter URL<\/li>\n<\/ul>\n

      5. Mengatur Akses File Statis<\/strong><\/p>\n

      Mengizinkan atau memblokir file:<\/p>\n

        \n
      • CSS<\/li>\n
      • JavaScript<\/li>\n
      • Gambar<\/li>\n<\/ul>\n

        Struktur Robots.txt dan Penjelasannya<\/strong><\/h2>\n

        Struktur robots.txt sebenarnya sangat sederhana dan mudah dipahami. Berikut contoh struktur dasar yang sering digunakan:<\/p>\n

        \n
        User<\/span>-agent: *
        \nDisallow: \/wp-admin<\/span>\/
        \nAllow: \/wp-admin<\/span>\/admin<\/span>-ajax.php
        \nSitemap: https:\/\/namadomain.com\/sitemap.xml
        \n<\/code><\/div>\n<\/div>\n

        Artinya bisa dijelaskan dengan bahasa yang lebih mudah berikut ini:<\/p>\n

          \n
        • User-agent: *<\/strong>
          Bagian ini berarti aturan berlaku untuk semua bot mesin pencari, seperti Google, Bing, dan lainnya.<\/li>\n
        • Disallow: \/wp-admin\/<\/strong>
          Perintah ini memberitahu bot agar tidak masuk ke folder wp-admin, karena folder ini berisi halaman admin yang tidak perlu tampil di Google.<\/li>\n
        • Allow: \/wp-admin\/admin-ajax.php<\/strong>
          Meskipun wp-admin diblokir, file ini tetap diizinkan diakses karena dibutuhkan agar fitur WordPress berjalan dengan baik.<\/li>\n
        • Sitemap: https:\/\/namadomain.com\/sitemap.xml<\/strong>
          Ini adalah alamat peta website (sitemap) yang membantu mesin pencari menemukan semua halaman penting dengan lebih cepat.<\/li>\n<\/ul>\n

          Contoh Robots.txt Berdasarkan Kebutuhan<\/strong><\/h2>\n

          1. WordPress Standar<\/strong><\/p>\n

          \n
          User-agent: *<\/span>
          \nDisallow: \/wp-admin\/<\/span>
          \nDisallow: \/wp-includes\/<\/span>
          \nAllow: \/wp-admin\/admin-ajax.php<\/span>
          \nSitemap: https:\/\/namadomain.com\/sitemap.xml<\/span>
          \n<\/code><\/div>\n<\/div>\n

          2. Website Maintenance<\/strong><\/p>\n

          \n
          User-agent: *<\/span>
          \nDisallow: \/<\/span>
          \n<\/code><\/div>\n<\/div>\n

          3. E-Commerce<\/strong><\/p>\n

          \n
          User-agent: *<\/span>
          \nDisallow: \/cart\/<\/span>
          \nDisallow: \/checkout\/<\/span>
          \nDisallow: \/my-account\/<\/span>
          \nAllow: \/<\/span>
          \n<\/code><\/div>\n<\/div>\n

          Cara Membuat dan Mengedit Robots.txt<\/strong><\/h2>\n

          1. Menggunakan Plugin SEO<\/strong><\/p>\n

          Yoast, Rank Math, All in One SEO menyediakan editor robots.txt langsung di dashboard.<\/p>\n

          2. Membuat Manual<\/strong><\/p>\n

          Melalui:<\/p>\n

            \n
          • File Manager cPanel<\/li>\n
          • FTP<\/li>\n<\/ul>\n

            Kesalahan Fatal dalam Robots.txt<\/strong><\/h2>\n
              \n
            • Memblokir seluruh website tanpa sadar<\/li>\n
            • Salah penulisan struktur<\/li>\n
            • Menutup file CSS & JS<\/li>\n
            • Tidak menambahkan sitemap<\/li>\n
            • Tidak menguji di Search Console<\/li>\n<\/ul>\n

              Cara Mengecek & Menguji Robots.txt<\/strong><\/h2>\n

              Gunakan:<\/p>\n

                \n
              • Google Search Console \u2192 Robots.txt Tester<\/li>\n
              • Browser: namadomain.com\/robots.txt<\/code><\/li>\n<\/ul>\n

                Kesimpulan<\/strong><\/h2>\n

                Robots.txt adalah fondasi penting dalam SEO teknis. Dengan memahami dan menerapkan robots.txt secara benar, kamu dapat mengontrol bagaimana mesin pencari menjelajahi website, mengoptimalkan crawl budget, melindungi halaman sensitif, dan meningkatkan performa SEO jangka panjang.<\/p>\n\n\n

                \n \n
                \n \n
                \n
                \n \n\n
                <\/div>\n <\/div>\n
                \n \n\n
                <\/div>\n <\/div>\n
                \n \n\n
                <\/div>\n <\/div>\n
                \n \n\n
                <\/div>\n <\/div>\n
                \n \n\n
                <\/div>\n <\/div>\n <\/div>\n \n
                \n
                \n \n\n
                <\/div>\n <\/div>\n
                \n \n\n
                <\/div>\n <\/div>\n
                \n \n\n
                <\/div>\n <\/div>\n
                \n \n\n
                <\/div>\n <\/div>\n
                \n \n\n
                <\/div>\n <\/div>\n <\/div>\n<\/div>\n \n\n
                \n 5\/5 - (1 vote) <\/div>\n <\/div>\n","protected":false},"excerpt":{"rendered":"

                Robots.txt adalah salah satu file paling penting dalam pengelolaan website dan SEO, meskipun ukurannya sangat kecil. File ini berfungsi sebagai \u201cpeta aturan\u201d yang memberi tahu mesin pencari bagaimana mereka boleh menjelajahi website kamu. Pengaturan robots.txt yang tepat akan membantu website lebih cepat terindeks, lebih aman, dan lebih optimal di mesin pencari. Artikel ini membahas secara […]<\/p>\n","protected":false},"author":13,"featured_media":29430,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"rop_custom_images_group":[],"rop_custom_messages_group":[],"rop_publish_now":"no","rop_publish_now_accounts":{"twitter_2392824914_2392824914":""},"rop_publish_now_history":[{"account":"twitter_2392824914_2392824914","service":"twitter","timestamp":1768637495,"status":"error"}],"rop_publish_now_status":"done","_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"enabled":false},"version":2}},"categories":[3,8],"tags":[14683,14685,14686,14682,14684],"class_list":["post-29425","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-blog","category-wordpress","tag-cara-membuat-robots-txt","tag-fungsi-robots-txt","tag-pengertian-robots-txt","tag-robots-txt-wordpress","tag-struktur-robots-txt"],"featured_image_src":{"landsacpe":["https:\/\/hosteko.com\/htk-blog\/wp-content\/uploads\/2026\/01\/Desain-tanpa-judul138-1140x445.png",1140,445,true],"list":["https:\/\/hosteko.com\/htk-blog\/wp-content\/uploads\/2026\/01\/Desain-tanpa-judul138-463x348.png",463,348,true],"medium":["https:\/\/hosteko.com\/htk-blog\/wp-content\/uploads\/2026\/01\/Desain-tanpa-judul138-300x169.png",300,169,true],"full":["https:\/\/hosteko.com\/htk-blog\/wp-content\/uploads\/2026\/01\/Desain-tanpa-judul138.png",1366,768,false]},"yoast_head":"\nApa Itu Robots.txt? Panduan Lengkap Fungsi, Struktur, dan Cara Membuatnya - Hosteko Blog<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/hosteko.com\/blog\/mengenal-robots-txt\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Apa Itu Robots.txt? Panduan Lengkap Fungsi, Struktur, dan Cara Membuatnya - Hosteko Blog\" \/>\n<meta property=\"og:description\" content=\"Robots.txt adalah salah satu file paling penting dalam pengelolaan website dan SEO, meskipun ukurannya sangat kecil. File ini berfungsi sebagai \u201cpeta aturan\u201d yang memberi tahu mesin pencari bagaimana mereka boleh menjelajahi website kamu. Pengaturan robots.txt yang tepat akan membantu website lebih cepat terindeks, lebih aman, dan lebih optimal di mesin pencari. Artikel ini membahas secara […]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/hosteko.com\/blog\/mengenal-robots-txt\" \/>\n<meta property=\"og:site_name\" content=\"Hosteko Blog\" \/>\n<meta property=\"article:published_time\" content=\"2026-01-17T08:11:24+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/hosteko.com\/htk-blog\/wp-content\/uploads\/2026\/01\/Desain-tanpa-judul138.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1366\" \/>\n\t<meta property=\"og:image:height\" content=\"768\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Fitri Ana\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Fitri Ana\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/hosteko.com\/blog\/mengenal-robots-txt#article\",\"isPartOf\":{\"@id\":\"https:\/\/hosteko.com\/blog\/mengenal-robots-txt\"},\"author\":{\"name\":\"Fitri Ana\",\"@id\":\"https:\/\/hosteko.com\/blog\/#\/schema\/person\/ffcd8071a8a3d6a862a4e1381d1c4ea0\"},\"headline\":\"Apa Itu Robots.txt? Panduan Lengkap Fungsi, Struktur, dan Cara Membuatnya\",\"datePublished\":\"2026-01-17T08:11:24+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/hosteko.com\/blog\/mengenal-robots-txt\"},\"wordCount\":653,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/hosteko.com\/blog\/#organization\"},\"image\":{\"@id\":\"https:\/\/hosteko.com\/blog\/mengenal-robots-txt#primaryimage\"},\"thumbnailUrl\":\"https:\/\/hosteko.com\/htk-blog\/wp-content\/uploads\/2026\/01\/Desain-tanpa-judul138.png\",\"keywords\":[\"cara membuat robots.txt\",\"fungsi robots.txt\",\"pengertian robots.txt\",\"robots txt WordPress\",\"struktur robots.txt\"],\"articleSection\":[\"Blog\",\"Wordpress\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/hosteko.com\/blog\/mengenal-robots-txt#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/hosteko.com\/blog\/mengenal-robots-txt\",\"url\":\"https:\/\/hosteko.com\/blog\/mengenal-robots-txt\",\"name\":\"Apa Itu Robots.txt? Panduan Lengkap Fungsi, Struktur, dan Cara Membuatnya - Hosteko Blog\",\"isPartOf\":{\"@id\":\"https:\/\/hosteko.com\/blog\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/hosteko.com\/blog\/mengenal-robots-txt#primaryimage\"},\"image\":{\"@id\":\"https:\/\/hosteko.com\/blog\/mengenal-robots-txt#primaryimage\"},\"thumbnailUrl\":\"https:\/\/hosteko.com\/htk-blog\/wp-content\/uploads\/2026\/01\/Desain-tanpa-judul138.png\",\"datePublished\":\"2026-01-17T08:11:24+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/hosteko.com\/blog\/mengenal-robots-txt#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/hosteko.com\/blog\/mengenal-robots-txt\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/hosteko.com\/blog\/mengenal-robots-txt#primaryimage\",\"url\":\"https:\/\/hosteko.com\/htk-blog\/wp-content\/uploads\/2026\/01\/Desain-tanpa-judul138.png\",\"contentUrl\":\"https:\/\/hosteko.com\/htk-blog\/wp-content\/uploads\/2026\/01\/Desain-tanpa-judul138.png\",\"width\":1366,\"height\":768},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/hosteko.com\/blog\/mengenal-robots-txt#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/hosteko.com\/blog\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Apa Itu Robots.txt? Panduan Lengkap Fungsi, Struktur, dan Cara Membuatnya\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/hosteko.com\/blog\/#website\",\"url\":\"https:\/\/hosteko.com\/blog\/\",\"name\":\"Hosteko Blog\",\"description\":\"Berita & Informasi Dunia IT\",\"publisher\":{\"@id\":\"https:\/\/hosteko.com\/blog\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/hosteko.com\/blog\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/hosteko.com\/blog\/#organization\",\"name\":\"HOSTEKO\",\"url\":\"https:\/\/hosteko.com\/blog\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/hosteko.com\/blog\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/hosteko.com\/htk-blog\/wp-content\/uploads\/2019\/04\/logo-hosteko.png\",\"contentUrl\":\"https:\/\/hosteko.com\/htk-blog\/wp-content\/uploads\/2019\/04\/logo-hosteko.png\",\"width\":195,\"height\":57,\"caption\":\"HOSTEKO\"},\"image\":{\"@id\":\"https:\/\/hosteko.com\/blog\/#\/schema\/logo\/image\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\/\/hosteko.com\/blog\/#\/schema\/person\/ffcd8071a8a3d6a862a4e1381d1c4ea0\",\"name\":\"Fitri Ana\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/hosteko.com\/blog\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/a099f723518ab9b89aa80b7ed8cadd2d3fae127c5bd735733cf24a661e75a882?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/a099f723518ab9b89aa80b7ed8cadd2d3fae127c5bd735733cf24a661e75a882?s=96&d=mm&r=g\",\"caption\":\"Fitri Ana\"}}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Apa Itu Robots.txt? Panduan Lengkap Fungsi, Struktur, dan Cara Membuatnya - Hosteko Blog","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/hosteko.com\/blog\/mengenal-robots-txt","og_locale":"en_US","og_type":"article","og_title":"Apa Itu Robots.txt? Panduan Lengkap Fungsi, Struktur, dan Cara Membuatnya - Hosteko Blog","og_description":"Robots.txt adalah salah satu file paling penting dalam pengelolaan website dan SEO, meskipun ukurannya sangat kecil. File ini berfungsi sebagai \u201cpeta aturan\u201d yang memberi tahu mesin pencari bagaimana mereka boleh menjelajahi website kamu. Pengaturan robots.txt yang tepat akan membantu website lebih cepat terindeks, lebih aman, dan lebih optimal di mesin pencari. Artikel ini membahas secara […]","og_url":"https:\/\/hosteko.com\/blog\/mengenal-robots-txt","og_site_name":"Hosteko Blog","article_published_time":"2026-01-17T08:11:24+00:00","og_image":[{"width":1366,"height":768,"url":"https:\/\/hosteko.com\/htk-blog\/wp-content\/uploads\/2026\/01\/Desain-tanpa-judul138.png","type":"image\/png"}],"author":"Fitri Ana","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Fitri Ana","Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/hosteko.com\/blog\/mengenal-robots-txt#article","isPartOf":{"@id":"https:\/\/hosteko.com\/blog\/mengenal-robots-txt"},"author":{"name":"Fitri Ana","@id":"https:\/\/hosteko.com\/blog\/#\/schema\/person\/ffcd8071a8a3d6a862a4e1381d1c4ea0"},"headline":"Apa Itu Robots.txt? Panduan Lengkap Fungsi, Struktur, dan Cara Membuatnya","datePublished":"2026-01-17T08:11:24+00:00","mainEntityOfPage":{"@id":"https:\/\/hosteko.com\/blog\/mengenal-robots-txt"},"wordCount":653,"commentCount":0,"publisher":{"@id":"https:\/\/hosteko.com\/blog\/#organization"},"image":{"@id":"https:\/\/hosteko.com\/blog\/mengenal-robots-txt#primaryimage"},"thumbnailUrl":"https:\/\/hosteko.com\/htk-blog\/wp-content\/uploads\/2026\/01\/Desain-tanpa-judul138.png","keywords":["cara membuat robots.txt","fungsi robots.txt","pengertian robots.txt","robots txt WordPress","struktur robots.txt"],"articleSection":["Blog","Wordpress"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/hosteko.com\/blog\/mengenal-robots-txt#respond"]}]},{"@type":"WebPage","@id":"https:\/\/hosteko.com\/blog\/mengenal-robots-txt","url":"https:\/\/hosteko.com\/blog\/mengenal-robots-txt","name":"Apa Itu Robots.txt? Panduan Lengkap Fungsi, Struktur, dan Cara Membuatnya - Hosteko Blog","isPartOf":{"@id":"https:\/\/hosteko.com\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/hosteko.com\/blog\/mengenal-robots-txt#primaryimage"},"image":{"@id":"https:\/\/hosteko.com\/blog\/mengenal-robots-txt#primaryimage"},"thumbnailUrl":"https:\/\/hosteko.com\/htk-blog\/wp-content\/uploads\/2026\/01\/Desain-tanpa-judul138.png","datePublished":"2026-01-17T08:11:24+00:00","breadcrumb":{"@id":"https:\/\/hosteko.com\/blog\/mengenal-robots-txt#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/hosteko.com\/blog\/mengenal-robots-txt"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/hosteko.com\/blog\/mengenal-robots-txt#primaryimage","url":"https:\/\/hosteko.com\/htk-blog\/wp-content\/uploads\/2026\/01\/Desain-tanpa-judul138.png","contentUrl":"https:\/\/hosteko.com\/htk-blog\/wp-content\/uploads\/2026\/01\/Desain-tanpa-judul138.png","width":1366,"height":768},{"@type":"BreadcrumbList","@id":"https:\/\/hosteko.com\/blog\/mengenal-robots-txt#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/hosteko.com\/blog"},{"@type":"ListItem","position":2,"name":"Apa Itu Robots.txt? Panduan Lengkap Fungsi, Struktur, dan Cara Membuatnya"}]},{"@type":"WebSite","@id":"https:\/\/hosteko.com\/blog\/#website","url":"https:\/\/hosteko.com\/blog\/","name":"Hosteko Blog","description":"Berita & Informasi Dunia IT","publisher":{"@id":"https:\/\/hosteko.com\/blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/hosteko.com\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/hosteko.com\/blog\/#organization","name":"HOSTEKO","url":"https:\/\/hosteko.com\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/hosteko.com\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/hosteko.com\/htk-blog\/wp-content\/uploads\/2019\/04\/logo-hosteko.png","contentUrl":"https:\/\/hosteko.com\/htk-blog\/wp-content\/uploads\/2019\/04\/logo-hosteko.png","width":195,"height":57,"caption":"HOSTEKO"},"image":{"@id":"https:\/\/hosteko.com\/blog\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/hosteko.com\/blog\/#\/schema\/person\/ffcd8071a8a3d6a862a4e1381d1c4ea0","name":"Fitri Ana","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/hosteko.com\/blog\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/a099f723518ab9b89aa80b7ed8cadd2d3fae127c5bd735733cf24a661e75a882?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/a099f723518ab9b89aa80b7ed8cadd2d3fae127c5bd735733cf24a661e75a882?s=96&d=mm&r=g","caption":"Fitri Ana"}}]}},"jetpack_publicize_connections":[],"jetpack_featured_media_url":"https:\/\/hosteko.com\/htk-blog\/wp-content\/uploads\/2026\/01\/Desain-tanpa-judul138.png","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/hosteko.com\/blog\/wp-json\/wp\/v2\/posts\/29425","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/hosteko.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/hosteko.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/hosteko.com\/blog\/wp-json\/wp\/v2\/users\/13"}],"replies":[{"embeddable":true,"href":"https:\/\/hosteko.com\/blog\/wp-json\/wp\/v2\/comments?post=29425"}],"version-history":[{"count":1,"href":"https:\/\/hosteko.com\/blog\/wp-json\/wp\/v2\/posts\/29425\/revisions"}],"predecessor-version":[{"id":29431,"href":"https:\/\/hosteko.com\/blog\/wp-json\/wp\/v2\/posts\/29425\/revisions\/29431"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/hosteko.com\/blog\/wp-json\/wp\/v2\/media\/29430"}],"wp:attachment":[{"href":"https:\/\/hosteko.com\/blog\/wp-json\/wp\/v2\/media?parent=29425"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/hosteko.com\/blog\/wp-json\/wp\/v2\/categories?post=29425"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/hosteko.com\/blog\/wp-json\/wp\/v2\/tags?post=29425"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}