Python Tutorial: Statistical Models

Описание: Want to learn more? Take the full course at https://learn.datacamp.com/courses/ad... at your own pace. More than a video, you'll learn hands-on coding & quickly apply skills to your daily work.

---

Let's add some more power to the NLP object!

In this video, you'll learn about spaCy's statistical models.

Some of the most interesting things you can analyze are context-specific: for example, whether a word is a verb or whether a span of text is a person name.

Statistical models enable spaCy to make predictions in context. This usually includes part-of-speech tags, syntactic dependencies and named entities.
Models are trained on large datasets of labeled example texts.

They can be updated with more examples to fine-tune their predictions – for example, to perform better on your specific data.

spaCy provides a number of pre-trained model packages you can download. For example, the "en_core_web_sm" package is a small English model that supports all core capabilities and is trained on web text.

The spacy dot load method loads a model package by name and returns an NLP object.

The package provides the binary weights that enable spaCy to make predictions.

It also includes the vocabulary and meta information to tell spaCy which language class to use and how to configure the processing pipeline.

Let's take a look at the model's predictions. In this example, we're using spaCy to predict part-of-speech tags, the word types in context.

First, we load the small English model and receive an NLP object.

Next, we're processing the text "She ate the pizza".

For each token in the Doc, we can print the text and the "pos underscore" attribute, the predicted part-of-speech tag.

In spaCy, attributes that return strings usually end with an underscore – attributes without the underscore return an ID.

Here, the model correctly predicted "ate" as a verb and "pizza" as a noun.

In addition to the part-of-speech tags, we can also predict how the words are related. For example, whether a word is the subject of the sentence or an object.

The "dep underscore" attribute returns the predicted dependency label.

The head attribute returns the syntactic head token. You can also think of it as the parent token this word is attached to.

To describe syntactic dependencies, spaCy uses a standardized label scheme. Here's an example of some common labels:

The pronoun "She" is a nominal subject attached to the verb – in this case, to "ate".

The noun "pizza" is a direct object attached to the verb "ate". It is eaten by the subject, "she".

The determiner "the", also known as an article, is attached to the noun "pizza".

Named entities are "real world objects" that are assigned a name – for example, a person, an organization or a country.

The doc dot ents property lets you access the named entities predicted by the model.

It returns an iterator of Span objects, so we can print the entity text and the entity label using the "label underscore" attribute.

In this case, the model is correctly predicting "Apple" as an organization, "U.K." as a geopolitical entity and "$1 billion" as money.

A quick tip: To get definitions for the most common tags and labels, you can use the spacy dot to explain the helper function.

For example, "GPE" for geopolitical entity isn't exactly intuitive – but spacy dot explain can tell you that it refers to countries, cities, and states.

The same works for part-of-speech tags and dependency labels.

Now it's your turn. Let's take a look at spaCy's statistical models and their predictions.


#DataCamp #PythonTutorial #AdvancedNLPwithspaCy #spaCy #PythonNLP #StatisticalModels