爬行器

Google的爬行器喜欢直接访问内容。

Google 's spider prefers to directly access content .

Google爬行器会由于元刷新重定向而阻塞，而302重定向会导致重复内容处罚。

The Google spider will choke on a meta refresh redirect and a302 redirect can cause duplicate content penalties .

X射线管道爬行器可编程序控制系统的研制

Development of Programmable Control System of Radiographic Crawler for Pipeline

X射线管道爬行器检测质量控制

Testing Quality Control of X Radiographic Crawler for Oil & Gas Pipeline

X射线管道爬行器的运行速度控制

The Speed Controller in the X-ray Pipe Crawler

改进了爬行器中动态搜索的Fish算法。

Make improvements in Fish algorithm , the dynamic search algorithm of integrated risk crawler .

同时，在借鉴传统搜索引擎架构设计的基础上，创造性地提出了支持聚焦搜索和隐藏Web搜索的策略可扩展的爬行器框架-Webob-Crawler。

And based on architecture of traditional search engines , we design an open crawler architecture that support focused crawling and hidden-web crawling .

就目前互联网的规模来说，单机版的网络爬行器已无法在一个有效的时间范围内完成一次搜集整个Web的任务。

On the current scale of the Internet , the single web crawler is unable to visit the entire web in an effective time-frame .

面向Blog的网络爬行器设计与实现

Design and Implementation on Blog-Oriented Web Crawler

如果使用的是web爬行器，那么可以通过在IBMHTTPServer上运行的本地站点中发布一个web页面实现。

If using a web crawler , this could be done by publishing a web page on a local site running on IBM HTTP Server .

维、哈、柯多文种搜索引擎中网页爬行器（Crawler）的设计与实现

Design and Implementation of Crawler of Uyghur , Kazak , Kyrgyz Multi-lingual Search Engine

参数的问题在于，爬行器可能受到URL字符的阻碍，比如与字符（&）。

The problem with parameters is that the spider might choke on URL characters such as ampersand ( & ) .

可以预见，创建一个大规模webhook系统，你将面临聚合器（aggregator）/爬行器（crawler）问题。

It works out that creating a big scale web hooks system you end up with aggregator / crawler problems .

冲突规避算法将URL散列和站点名散列相结合，应用多线程技术，在实现了负载平衡同时，又避免了并行爬行器的并发冲突。

The collision avoidance algorithm combines URL hashing and site-name hashing . It realizes work load balancing and also avoids the collision when parallel fetching .

如果使用Frame、JavaScript和页面阅读器无法读取的其他技术，也会产生爬行器无法读取（或者会阻碍爬行器）的内容。

Using Frames , JavaScript , and other technology that a page reader can 't read also creates content a spider can 't read ( or will choke on ) .

爬行器如果按照传统的宽度优先搜索策略在Web空间中爬行，它对所有的网页都采取一视同仁的态度，这样爬行的结果就导致了它所爬行回来的网页质量不高。

Crawling in Web space according to the strategy of the traditional breadth-first search , if a crawler respects the importance of every page equally , the quality of Web pages collected by the crawler is not high .

用户最初查看Web站点的方式与爬行器的方式相同，所以按照自左上角向下强调关键词也是一种良好的Web设计实践（参见视线跟踪和搜索行为）。

Users will initially view your Web site the same way the spider does , so emphasizing the keywords from the top-left-down is a good Web design practice as well ( see Eye tracking and search behavior ) .

服务器重定向向Google爬行器表明这个页面永久转移了，应该将301另一端的页面作为新URL对待。

A server redirect tells the Google spider that the page is moved permanently and to treat the page at the other end of the301 as the new URL .

接下来详细论述了TSWS三个主要部分HTML解析器、Web爬行器和文本分类器的设计与实现。

Next , this paper presents the design and implementation of TSWS , which includes HTML Parser , Topic-Specific Web Crawler and Web Classifier .

假设SCA和MDB应用程序已经部署并启动，确保针对特定文档集合的ICA爬行器和索引器均已运行。

Assuming that SCA and MDB applications were already deployed and started , ensure that the ICA crawler and indexer for a particular document collection are running .

创建合理的关键词策略是最重要的SEO任务，这可以满足爬行器和潜在受众的需要。

Creating a well-reasoned keyword strategy is the most important SEO task you can perform to meet both the needs of spiders and your potential audience .

详细阐述了用于测量Gnutella网络拓扑的爬行器的设计和实现，并实验性地测定了完成该网络拓扑快照的最佳快门延迟。

The paper designs and implements a new crawler to extract the topology of the Gnutella network , and empirically deduces the shutter speed for an accurate topological snapshot .

分析和设计了一个合作式主题爬行器。

This dissertation analyses and designs a collaborative focused crawler model .

一种可用于管道爬行器的静磁场型定位系统

A New Static Magnetic Intersect System Used in Tube Creeper

细小管道爬行器的研究与开发

The Research and Development of Crawl Device in Small Pipe

管道对接焊缝超声检测爬行器的研制

Automatic scan device of ultrasonic NDT in detecting the pipeline girth welds

海底石油管道缺陷检测爬行器的运动控制

Movement Control of Creeper for Offshore Oilfield Pipeline Defect Detection

杆（管）件自动爬行器的结构分析与比较

Analysis and Compare of Automatic Pole ( Pipeline ) - Crawling Device

利用超链接信息改进网页爬行器的搜索策略

Using Hyperlink Information to Improve Crawler 's Searching Strategy

γ射线管道爬行器的研制

Development of γ - ray Crawler for Pipeline Inspection