Fulltext Index Study2：Pupulate

Posted 2020-07-16 悦光阴

Creating and maintaining a full-text index involves populating the index by using a process called a population (also known as a crawl).

由于创建Fulltext Index 会消耗大量的系统资源，因此Fulltext Index 必须在系统空间的时间进行maintain和crawl。在创建Fulltext Index时，通过指定 CHANGE_TRACKING= MANUAL，或 CHANGE_TRACKING= OFF, NO POPULATION，这样Fulltext Index 不会在创建时立即crawl。

在系统空闲时，使用 alter fulltext index 语句crawl。只有在crawl之后，fulltext index 才能使用。

ALTER FULLTEXT INDEX ON table_name
START { FULL | INCREMENTAL | UPDATE } POPULATION;

在创建 Fulltext Index时，如果指定CHANGE_TRACKING=AUTO   或   CHANGE_TRACKING=  OFF ， fulltext index在创建之后，立即crawl。    

CHANGE_TRACKING [ = ] { MANUAL | AUTO | OFF [ , NO POPULATION ] }              

Specifies whether changes (updates, deletes or inserts) made to table columns that are covered by the full-text index will be propagated by SQL Server to the full-text index. Data changes through WRITETEXT and UPDATETEXT are not reflected in the full-text index, and are not picked up with change tracking.

MANUAL                   

Specifies that the tracked changes must be propagated manually by calling the ALTER FULLTEXT INDEX … START UPDATE POPULATION Transact-SQL statement (manual population). You can use SQL Server Agent to call this Transact-SQL statement periodically.

AUTO                  

Specifies that the tracked changes will be propagated automatically as data is modified in the base table (automatic population). Although changes are propagated automatically, these changes might not be reflected immediately in the full-text index. AUTO is the default.

OFF [ , NO POPULATION]                   

Specifies that SQL Server does not keep a list of changes to the indexed data. When NO POPULATION is not specified, SQL Server populates the index fully after it is created.

The NO POPULATION option can be used only when CHANGE_TRACKING is OFF. When NO POPULATION is specified, SQL Server does not populate an index after it is created. The index is only populated after the user executes the ALTER FULLTEXT INDEX command with the START FULL POPULATION or START INCREMENTAL POPULATION clause.

 

1，Full Population

During a full population, index entries are built for all the rows of a table or indexed view. A full population of a full-text index, builds index entries for all the rows of the base table or indexed view.

By default, SQL Server populates a new full-text index fully as soon as it is created. However, a full population can consume a significant amount of resources. Therefore, when creating a full-text index during peak periods, it is often a best practice to delay the full population until an off-peak time, particularly if the base table of an full-text index is large. However, the full-text catalog to which the index belongs is not usable until all of its full-text indexes are populated. To create a full-text index without populating it immediately, specify the CHANGE_TRACKING OFF, NO POPULATION clause in the CREATE FULLTEXT INDEX statement. If you specify CHANGE_TRACKING MANUAL, the Full-Text Engine uses statement and SQL Server will not populate the new full-text index until you execute an ALTER FULLTEXT INDEX statement using the START FULL POPULATION or START INCREMENTAL POPULATION clause.

To run an full population

ALTER FULLTEXT INDEX 
ON [dbo].[DatabaseLog]
   START FULL POPULATION;

2，Change Tracking-Based Population

Optionally, you can use change tracking to maintain a full-text index after its initial full population. There is a small overhead associated with change tracking because SQL Server maintains a table in which it tracks changes to the base table since the last population. When change tracking is used, SQL Server maintains a record of the rows in the base table or indexed view that have been modified by updates, deletes, or inserts. Data changes through WRITETEXT and UPDATETEXT are not reflected in the full-text index, and are not picked up with change tracking.

When change tracking is enabled during index creation, SQL Server fully populates the new full-text index immediately after it is created. Thereafter, changes are tracked and propagated to the full-text index. There are two types of change tracking, automatic (CHANGE_TRACKING AUTO option) and manual (CHANGE_TRACKING MANUAL option). Automatic change tracking is the default behavior.

The type of change tracking determines how the full-text index is populated, as follows:

• Automatic population

  By default, or if you specify CHANGE_TRACKING AUTO, the Full-Text Engine uses automatic population on the full-text index. After the initial full population completes, changes are tracked as data is modified in the base table, and the tracked changes are propagated automatically. The full-text index is updated in the background, however, so propagated changes might not be reflected immediately in the index.

• Manual population

If you specify CHANGE_TRACKING MANUAL, the Full-Text Engine uses manual population on the full-text index. After the initial full population completes, changes are tracked as data is modified in the base table. However, they are not propagated to the full-text index until you execute an ALTER FULLTEXT INDEX … START UPDATE POPULATION statement. You can use SQL Server Agent to call this Transact-SQL statement periodically.

To turn on change tracking

• CREATE FULLTEXT INDEX … WITH CHANGE_TRACKING AUTO | Manual
• ALTER FULLTEXT INDEX … SET CHANGE_TRACKING AUTO  | Manual

To turn off change tracking          

• CREATE FULLTEXT INDEX … WITH CHANGE_TRACKING OFF

• ALTER FULLTEXT INDEX … SET CHANGE_TRACKING OFF

To run an update population

ALTER FULLTEXT INDEX 
ON [dbo].[DatabaseLog]
   START UPDATE POPULATION;

3,Incremental Timestamp-Based Population

An incremental population is an alternative mechanism for manually populating a full-text index. You can run an incremental population for a full-text index that has CHANGE_TRACKING set to MANUAL or OFF. If the first population on a full-text index is an incremental population, it indexes all rows, making it equivalent to a full population.

The requirement for incremental population is that the indexed table must have a column of the timestamp data type. If a timestamp column does not exist, incremental population cannot be performed. A request for incremental population on a table without a timestamp column results in a full population operation. Also, if any metadata that affects the full-text index for the table has changed since the last population, incremental population requests are implemented as full populations. This includes metadata changes caused by altering any column, index, or full-text index definitions.

SQL Server uses the timestamp column to identify rows that have changed since the last population. The incremental population then updates the full-text index for rows added, deleted, or modified after the last population, or while the last population was in progress. If a table experiences a high volume of inserts, using incremental population can be more efficient that using manual population.

At the end of a population, the Full-Text Engine records a new timestamp value. This value is the largest timestamp value that SQL Gatherer has encountered. This value will be used when a subsequent incremental population starts.

To run an incremental population, execute an ALTER FULLTEXT INDEX statement using the START INCREMENTAL POPULATION clause.

ALTER FULLTEXT INDEX 
ON [dbo].[DatabaseLog]
   START INCREMENTAL  POPULATION;

4,Creating a full-text index without running a full population

CREATE UNIQUE INDEX ui_ukDoc ON Production.Document(DocumentID);
CREATE FULLTEXT CATALOG AW_Production_FTCat;
CREATE FULLTEXT INDEX ON Production.Document
(
    Document                         --Full-text index column name 
        TYPE COLUMN FileExtension    --Name of column that contains file type information
        Language 1033                 --1033 is LCID for the English language
)
    KEY INDEX ui_ukDoc
    ON AW_Production_FTCat
    WITH CHANGE_TRACKING = OFF, NO POPULATION;
GO

ALTER FULLTEXT INDEX ON Production.Document
   START FULL POPULATION;
GO

参考doc：Populate Full-Text Indexes