Bot AI Crawling Isi Website

Version 2.0

Soeltansyah, Hilfan, 2024, "Bot AI Crawling Isi Website", https://doi.org/10.34820/FK2/LSY890, Telkom University Dataverse, V2, UNF:6:QwlWTTEN/m46HVMVGLVTmw== [fileUNF]

Learn about Data Citation Standards.

Contact Owner

Dataset Metrics

16 Downloads

Description	Semenjak 27 November 2025, di Cloudflare sudah ada Beta laporan AI Audit, dan dari grafik traficnya Bot AI mulai mendominasi untuk melakukan crawling isi website, tentu ini jadi tambahan beban bagi server hosting website. (2024-11-4)
Subject	Computer and Information Science
Keyword	Generative Engine Optimization
Related Publication	https://hilfan.staff.telkomuniversity.ac.id/bot-ai-mulai-mendominasi-untuk-melakukan-crawling-isi-website/
Notes	GEO adalah singkatan dari Generative Engine Optimization. Ini adalah proses untuk memastikan konten digital Anda memaksimalkan jangkauan dan visibilitasnya di dalam Generative AI Engine seperti ChatGPT, Claude, SGE, Gemini, Perplexity, dan lainnya saat orang bertanya

Find

Filter by

	1 to 4 of 4 Files	Original Format Archival Format (.tab)
	2024-11-05T06_16_31.456Z__2024-12-05T06_16_31.456Z.tab Tabular Data - 654.0 KB - Dec 5, 2024 - 7 Downloads 6 Variables, 4000 Observations - UNF:6:QwlWTTEN/m46HVMVGLVTmw== Tulisan detail Bot AI Mulai mendominasi untuk melakukan Crawling Isi Website https://hilfan.staff.telkomuniversity.ac.id/bot-ai-mulai-mendominasi-untuk-melakukan-crawling-isi-website/	Access File Download Options Original File Format (Comma Separated Values) Tab-Delimited RData Format Variable Metadata Data File Citation RIS EndNote XML BibTeX
	agent Bot AI crawling website.jpg JPEG Image - 68.8 KB - Dec 5, 2024 - 2 Downloads MD5: bdd3fb2cbd44638ea6e9eb3debada04e Para agent Bot AI crawling website https://hilfan.staff.telkomuniversity.ac.id/bot-ai-mulai-mendominasi-untuk-melakukan-crawling-isi-website/	Access File Download Options Download
	Bot AI crawling website.jpg JPEG Image - 92.0 KB - Dec 5, 2024 - 3 Downloads MD5: 9fb28c1c89ed51d74b915a87d0f16a45 Semenjak 27 November 2025, di Cloudflare sudah ada Beta laporan AI Audit, dan dari grafik traficnya Bot AI mulai mendominasi untuk melakukan crawling isi website, tentu ini jadi tambahan beban bagi server hosting website. https://hilfan.staff.telkomuniversity.ac.id/bot-ai-mulai-mendominasi-untuk-melakukan-crawling-isi-website/	Access File Download Options Download
	data yang digali untuk diambil oleh agent Bot AI crawling website.jpg JPEG Image - 127.1 KB - Dec 5, 2024 - 4 Downloads MD5: afd429cd78c392a6a049e76d0bc5cae1 data yang digali untuk diambil oleh agent Bot AI crawling website https://hilfan.staff.telkomuniversity.ac.id/bot-ai-mulai-mendominasi-untuk-melakukan-crawling-isi-website/	Access File Download Options Download

Citation Metadata

Dataset Persistent ID	doi:10.34820/FK2/LSY890
Publication Date	2024-12-05
Title	Bot AI Crawling Isi Website
Alternative Title	Data Set AI Audit
Alternative URL	https://hilfan.staff.telkomuniversity.ac.id/bot-ai-mulai-mendominasi-untuk-melakukan-crawling-isi-website/
Author	Soeltansyah, Hilfan (Telkom University - Direktorat Pusat Teknologi Informasi) - ORCID: 0009-0002-6152-8696
Contact	Use email button above to contact. Soeltansyah, Hilfan (Telkom University - Direktorat Pusat Teknologi Informasi)
Description	Semenjak 27 November 2025, di Cloudflare sudah ada Beta laporan AI Audit, dan dari grafik traficnya Bot AI mulai mendominasi untuk melakukan crawling isi website, tentu ini jadi tambahan beban bagi server hosting website. (2024-11-4)
Subject	Computer and Information Science
Keyword	Generative Engine Optimization (GEO) https://hilfan.staff.telkomuniversity.ac.id/apa-itu-generative-engine-optimization-geo-cara-melakukannya/
Topic Classification	AI Search, AI Crawler, AI Assistant (Bot AI) https://hilfan.staff.telkomuniversity.ac.id/bot-ai-mulai-mendominasi-untuk-melakukan-crawling-isi-website/
Related Publication	https://hilfan.staff.telkomuniversity.ac.id/bot-ai-mulai-mendominasi-untuk-melakukan-crawling-isi-website/
Notes	GEO adalah singkatan dari Generative Engine Optimization. Ini adalah proses untuk memastikan konten digital Anda memaksimalkan jangkauan dan visibilitasnya di dalam Generative AI Engine seperti ChatGPT, Claude, SGE, Gemini, Perplexity, dan lainnya saat orang bertanya
Distributor	Telkom University (Telkom University) (Tel-U) https://telkomuniversity.ac.id/
Depositor	Soeltansyah, Hilfan
Deposit Date	2024-12-05

Waiver

Our Community Norms as well as good scientific practices expect that proper credit is given via citation. Please use the data citation above, generated by the Dataverse.

CC0 - "Public Domain Dedication" Creative Commons CC0 1.0 Public Domain Dedication icon

Guestbook

No guestbook is assigned to this dataset, you will not be prompted to provide any information on file download.

	Dataset	Summary	Contributors	Published
No records found.

Restrict Files and Add Dataset Terms of Access

Restricting limits access to published files. You can add or edit Terms of Access for the dataset, and allow people to Request Access to restricted files.

Terms of Access

Request Access

Enable access request

Delete Files

The file will be deleted after you click on the Delete button.

Files will not be removed from previously published versions of the dataset.

Select File(s)

Please select one or more files.

Share Dataset

Share this dataset on your favorite social media networks.

Dataset Citations

Citations for this dataset are retrieved from Crossref via DataCite using Make Data Count standards. For more information about dataset metrics, please refer to the User Guide.

Sorry, no citations were found.

Restricted Files Selected

The restricted file(s) selected may not be downloaded because you have not been granted access.

Restricted Files Selected

The restricted file(s) selected may not be downloaded because you have not been granted access.

Click Continue to download the files you have access to download.

Restricted Files Selected

The restricted file(s) selected may not be downloaded because you have not been granted access.

Click Continue to download the files you have access to download.

Delete Dataset

Are you sure you want to delete this dataset and all of its files? You cannot undelete this dataset.

Delete Draft Version

Are you sure you want to delete this draft version? Files will be reverted to the most recently published version. You cannot undelete this draft.

Unpublished Dataset Private URL

Private URL can only be used with unpublished versions of datasets.

Unpublished Dataset Private URL

Are you sure you want to disable the Private URL? If you have shared the Private URL with others they will no longer be able to use it to access your unpublished dataset.

Delete Files

The file(s) will be deleted after you click on the Delete button.

Files will not be removed from previously published versions of the dataset.

Compute

This dataset contains restricted files you may not compute on because you have not been granted access.

Deaccession Dataset

Are you sure you want to deaccession? The selected version(s) will no longer be viewable by the public.

Deaccession Dataset

Are you sure you want to deaccession this dataset? It will no longer be viewable by the public.

Version Differences Details

Please select two versions to view the differences.

Version Differences Details

Version:
Last Updated:

Select File(s)

Please select a file or files to be downloaded.

Select File(s)

Please select a file or files for access request.

Edit Tags

Select existing file tags or create new tags to describe your files. Each file can have more than one tag.

Request Access

You need to Sign Up or Log In to request access to this file.

Data Not Published

In order to map your data with WorldMap, your data must be published. Please publish this dataset, then retry the Map Data feature.

Dataset Terms

Please confirm and/or complete the information needed below in order to continue.

Preview Guestbook

Upon downloading files the guestbook asks for the following information.

Guestbook Name

Collected Data

Account Information

Package File Download

Use the Download URL in a Wget command or a download manager to download this package file. Download via web browser is not recommended. User Guide - Downloading a Dataverse Package via URL

- -

Download URL

https://dataverse.telkomuniversity.ac.id/api/access/datafile/

Request Access

Please confirm and/or complete the information needed below in order to request access to files in this dataset.

Compute Batch

Clear Batch

Dataset	Dataset Persistent ID

Compute Batch

Submit for Review

You will not be able to make changes to this dataset while it is in review.

Publish Dataset

Are you sure you want to republish this dataset?

Select if this is a minor or major version update.

Minor Release (2.1)

Major Release (3.0)

Publish Dataset

This dataset cannot be published until Telkom University Dataverse is published by its administrator.

Publish Dataset

This dataset cannot be published until Telkom University Dataverse and are published.

Return to Author

Return this dataset to contributor for modification.