Bot AI Crawling Isi Website (doi:10.34820/FK2/LSY890)
(Data Set AI Audit)

View:

Part 1: Document Description
Part 2: Study Description
Part 3: Data Files Description
Part 4: Variable Description
Part 5: Other Study-Related Materials
Entire Codebook

Document Description

Citation

Title:

Bot AI Crawling Isi Website

Identification Number:

doi:10.34820/FK2/LSY890

Distributor:

Telkom University Dataverse

Date of Distribution:

2024-12-05

Version:

2

Bibliographic Citation:

Soeltansyah, Hilfan, 2024, "Bot AI Crawling Isi Website", https://doi.org/10.34820/FK2/LSY890, Telkom University Dataverse, V2, UNF:6:QwlWTTEN/m46HVMVGLVTmw== [fileUNF]

Study Description

Citation

Title:

Bot AI Crawling Isi Website

Alternative Title:

Data Set AI Audit

Identification Number:

doi:10.34820/FK2/LSY890

Authoring Entity:

Soeltansyah, Hilfan (Telkom University - Direktorat Pusat Teknologi Informasi)

Distributor:

Telkom University Dataverse

Distributor:

Telkom University

Access Authority:

Soeltansyah, Hilfan

Depositor:

Soeltansyah, Hilfan

Date of Deposit:

2024-12-05

Study Scope

Keywords:

Computer and Information Science, Generative Engine Optimization

Topic Classification:

AI Search, AI Crawler, AI Assistant

Abstract:

Semenjak 27 November 2025, di Cloudflare sudah ada Beta laporan AI Audit, dan dari grafik traficnya Bot AI mulai mendominasi untuk melakukan crawling isi website, tentu ini jadi tambahan beban bagi server hosting website.

Notes:

GEO adalah singkatan dari Generative Engine Optimization. Ini adalah proses untuk memastikan konten digital Anda memaksimalkan jangkauan dan visibilitasnya di dalam Generative AI Engine seperti ChatGPT, Claude, SGE, Gemini, Perplexity, dan lainnya saat orang bertanya

Methodology and Processing

Sources Statement

Data Access

Notes:

CC0 Waiver

Other Study Description Materials

Related Publications

Citation

Bibliographic Citation:

https://hilfan.staff.telkomuniversity.ac.id/bot-ai-mulai-mendominasi-untuk-melakukan-crawling-isi-website/

File Description--f18472

File: 2024-11-05T06_16_31.456Z__2024-12-05T06_16_31.456Z.tab

  • Number of cases: 4000

  • No. of variables per record: 6

  • Type of File: text/tab-separated-values

Notes:

UNF:6:QwlWTTEN/m46HVMVGLVTmw==

Variable Description

List of Variables:

Variables

path

f18472 Location:

Variable Format: character

Notes: UNF:6:3LwEBQp3CuMd1UFbrzFlSA==

count

f18472 Location:

Summary Statistics: Valid 4000.0; Max. 1020.0; StDev 16.276887561958343; Mean 2.226750000000088; Min. 1.0

Variable Format: numeric

Notes: UNF:6:Nn/yK8JlVW10gAHEWeIZQw==

provider

f18472 Location:

Variable Format: character

Notes: UNF:6:xp/wsSWqoAG3RbtBGMl1dw==

pathType

f18472 Location:

Variable Format: character

Notes: UNF:6:6wlu7BpxcZXVDAGjtTG2vA==

type

f18472 Location:

Variable Format: character

Notes: UNF:6:Z+VVAQBPdPmH93gyJwINzw==

bot

f18472 Location:

Variable Format: character

Notes: UNF:6:hmp6dggH/cLKXbVnoAKEhw==

Other Study-Related Materials

Label:

agent Bot AI crawling website.jpg

Text:

Para agent Bot AI crawling website https://hilfan.staff.telkomuniversity.ac.id/bot-ai-mulai-mendominasi-untuk-melakukan-crawling-isi-website/

Notes:

image/jpeg

Other Study-Related Materials

Label:

Bot AI crawling website.jpg

Text:

Semenjak 27 November 2025, di Cloudflare sudah ada Beta laporan AI Audit, dan dari grafik traficnya Bot AI mulai mendominasi untuk melakukan crawling isi website, tentu ini jadi tambahan beban bagi server hosting website. https://hilfan.staff.telkomuniversity.ac.id/bot-ai-mulai-mendominasi-untuk-melakukan-crawling-isi-website/

Notes:

image/jpeg

Other Study-Related Materials

Label:

data yang digali untuk diambil oleh agent Bot AI crawling website.jpg

Text:

data yang digali untuk diambil oleh agent Bot AI crawling website https://hilfan.staff.telkomuniversity.ac.id/bot-ai-mulai-mendominasi-untuk-melakukan-crawling-isi-website/

Notes:

image/jpeg