Fulltext Index Study1：Usage

Posted 2020-07-16

一，在创建Fulltext Index的table上，必须使用Key Index（unique, single-key, non-nullable column）

CREATE UNIQUE INDEX ui_dbLogID 
ON [dbo].[DatabaseLog]([DatabaseLogID]);

The KEY INDEX must be a unique, single-key, non-nullable column. Select the smallest unique key index for the full-text unique key. For the best performance, we recommend an integer data type for the full-text key.

二，创建Fulltext Catalog，一个Fulltext Index 必须存在于一个Catalog中

One full-text catalog can have several full-text indexes, but a full-text index can only be part of one full-text catalog. Each database can contain zero or more full-text catalogs. The catalog name must be unique among all catalog names in the current database.

create fulltext catalog catalog_test
as default;

参考doc：CREATE FULLTEXT CATALOG (Transact-SQL)

三，CREATE FULLTEXT INDEX

Only one full-text index is allowed per table or indexed view, and each full-text index applies to a single table or indexed view.

create fulltext index 
on [dbo].[DatabaseLog]
(
[TSQL] language 1033
)
key index ui_dbLogID
on (catalog_test,FILEGROUP [primary])    --filegroup
with
( CHANGE_TRACKING=off ,NO POPULATION );

参考doc：CREATE FULLTEXT INDEX (Transact-SQL)

CREATE FULLTEXT INDEX ON table_name
   [ ( { column_name 
             [ TYPE COLUMN type_column_name ]
             [ LANGUAGE language_term ] 
             [ STATISTICAL_SEMANTICS ]
        } [ ,...n] 
      ) ]
    KEY INDEX index_name 
    [ ON <catalog_filegroup_option> ]
    [ WITH [ ( ] <with_option> [ ,...n] [ ) ] ]
[;]

<catalog_filegroup_option>::=
 {
    fulltext_catalog_name 
 | ( fulltext_catalog_name, FILEGROUP filegroup_name )
 | ( FILEGROUP filegroup_name, fulltext_catalog_name )
 | ( FILEGROUP filegroup_name )
 }

<with_option>::=
 {
   CHANGE_TRACKING [ = ] { MANUAL | AUTO | OFF [, NO POPULATION ] } 
 | STOPLIST [ = ] { OFF | SYSTEM | stoplist_name }
 | SEARCH PROPERTY LIST [ = ] property_list_name 
 }

1，type column type_column_name

　　This column, known as the type column, contains a user-supplied file extension (.doc, .pdf, .xls, and so forth).

　　Table中的 一个column，用于存储 file extension（文件扩展名）。

2，language，When creating a full-text index, you need to specify a column-level language for the indexed column.

language_term is optional and can be specified as a string, integer, or hexadecimal value corresponding to the locale identifier (LCID) of a language. If no value is specified, the default language of the SQL Server instance is used.

从sys.fulltext_languages 系统视图中查看language的LCID 和 LanguageName。

 

参考doc： Choose a Language When Creating a Full-Text Index

How Full-Text Search Uses the Name of the Column-Level Language

When creating a full-text index, you need to specify a valid language name for each column. If a language name is valid but not returned by the sys.fulltext_languages (Transact-SQL) catalog view, full-text search falls back to the closest available language name of the same language family, if any. Otherwise, full-text search falls back to the Neutral word breaker. This fall-back behavior might affect the recall accuracy. Therefore we strongly recommend that you specify a valid and available language name for each column when creating a full-text index.

The LCID is used against all data types eligible for full-text indexing (such as char or nchar). If you have the sort order of a char, varchar, or text type column set to a language setting different from the language identified by the LCID, the LCID is used anyway during full-text indexing and querying of those columns.

 

2，Populate，更新Fulltext Index 的方式

CHANGE_TRACKING [ = ] { MANUAL | AUTO | OFF [ , NO POPULATION ] }              

Specifies whether changes (updates, deletes or inserts) made to table columns that are covered by the full-text index will be propagated by SQL Server to the full-text index. Data changes through WRITETEXT and UPDATETEXT are not reflected in the full-text index, and are not picked up with change tracking.

MANUAL                   

Specifies that the tracked changes must be propagated manually by calling the ALTER FULLTEXT INDEX … START UPDATE POPULATION Transact-SQL statement (manual population). You can use SQL Server Agent to call this Transact-SQL statement periodically.

AUTO                  

Specifies that the tracked changes will be propagated automatically as data is modified in the base table (automatic population). Although changes are propagated automatically, these changes might not be reflected immediately in the full-text index. AUTO is the default.

OFF [ , NO POPULATION]                   

Specifies that SQL Server does not keep a list of changes to the indexed data. When NO POPULATION is not specified, SQL Server populates the index fully after it is created.

The NO POPULATION option can be used only when CHANGE_TRACKING is OFF. When NO POPULATION is specified, SQL Server does not populate an index after it is created. The index is only populated after the user executes the ALTER FULLTEXT INDEX command with the START FULL POPULATION or START INCREMENTAL POPULATION clause.

 3，STOPLIST

STOPLIST [ = ] { OFF | SYSTEM | stoplist_name }               

Associates a full-text stoplist with the index. The index is not populated with any tokens that are part of the specified stoplist. If STOPLIST is not specified, SQL Server associates the system full-text stoplist with the index. 

OFF                  

Specifies that no stoplist be associated with the full-text index.

SYSTEM                   

Specifies that the default full-text system STOPLIST should be used for this full-text index. 

stoplist_name                                       

Specifies the name of the stoplist to be associated with the full-text index.

4，SEARCH PROPERTY LIST

SEARCH PROPERTY LIST [ = ] property_list_name              

Associates a search property list with the index.

OFF                  

Specifies that no property list be associated with the full-text index.

property_list_name                                       

Specifies the name of the search property list to associate with the full-text index.

 

四，修改Fulltext Index，增加一个column

alter fulltext index 
on [dbo].[DatabaseLog]
add ([XmlEvent] language 1033)
WITH NO POPULATION;

参考doc：ALTER FULLTEXT INDEX (Transact-SQL)

ALTER FULLTEXT INDEX ON table_name
   { ENABLE 
   | DISABLE
   | SET CHANGE_TRACKING [ = ] { MANUAL | AUTO | OFF }
   | ADD ( column_name 
     [ TYPE COLUMN type_column_name ] 
     [ LANGUAGE language_term ]
     [ STATISTICAL_SEMANTICS ]
 [,...n] )
     [ WITH NO POPULATION ]
   | ALTER COLUMN column_name
     { ADD | DROP } STATISTICAL_SEMANTICS
     [ WITH NO POPULATION ]
   | DROP ( column_name [,...n] )
     [ WITH NO POPULATION ] 
   | START { FULL | INCREMENTAL | UPDATE } POPULATION
   | {STOP | PAUSE | RESUME } POPULATION 
   | SET STOPLIST [ = ] { OFF| SYSTEM | stoplist_name }
     [ WITH NO POPULATION ] 
   | SET SEARCH PROPERTY LIST [ = ] { OFF | property_list_name }
     [ WITH NO POPULATION ] 
   }
[;]

五，Populate Fulltext Index

ALTER FULLTEXT INDEX 
ON [dbo].[DatabaseLog]
   START FULL POPULATION;

六，使用 contains 查询 Fulltext Index

如果想要在查询中使用Fulltext Index，必须使用关键字：CONTAINS、FULLTEXT、CONTAINSTABLE、FREETEXTTABLE 告知 SQL Server Engine。

DECLARE @SearchWord nvarchar(30)
SET @SearchWord = N‘Error‘
SELECT [TSQL] 
FROM [dbo].[DatabaseLog] 
WHERE CONTAINS([TSQL], @SearchWord);

 

参考doc：CONTAINS (Transact-SQL)

CONTAINS ( 
     { 
        column_name | ( column_list ) 
      | * 
      | PROPERTY ( { column_name }, ‘property_name‘ )  
     } 
     , ‘<contains_search_condition>‘
     [ , LANGUAGE language_term ]
   ) 

使用Language language_term 制定一个language.

Specifying a Non-default Column-Level Language in a Full-Text Query

By default, in , full-text search will parse the query terms using the language specified for each column that is included in the full-text clause. To override this behavior, specify a nondefault language at query time. For supported languages whose resources are installed, the LANGUAGE language_term clause of a CONTAINS, CONTAINSTABLE, FREETEXT, or FREETEXTTABLE query can be used to specify the language used for word breaking, stemming, thesaurus, and stopword processing of the query terms.

七，Drop Fulltext Index

DROP FULLTEXT INDEX 
ON [dbo].[DatabaseLog]