Building Domain Specific Language Models

N-gram model, RNN, LSTM, AllenNLP

We just launched our liveProject platform — where you can sign up for a structured project and get real-world experience.

In this liveProject, you’ll step into the role of a natural language processing data scientist working for Stack Exchange. Stack Exchange runs a network of question-and-answer sites on diverse topics ranging from programming to cooking. Your boss wants you to create language models that are tuned to the particular vocabulary of different Stack Exchange sites. Language is domain specific, for example an insurance company’s documents will use very different terminology than a post on a social media site. Because of this, off-the-shelf NLP models trained on generic text can be inaccurate for specialized domains. Your goal is to build a language model capable of query completion, text generation, and sentence selection for the domain-specific language of the Cross Validated statistics and machine learning site. Challenges you will tackle include preparing your datasets, building and evaluating n-gram word-based language models, and building a character-based language model with AllenNLP.

Learn more about liveProject here:




Follow Manning Publications on Medium for free content and exclusive discounts.

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

How to implement Gradient Descent in Python

How to Create your own Sign Language Translation App by extending SigNN

Computer Vision : A short beginner’s guide.

AI supported B&W image colourisation with in 10 lines of code (well, kind of)

My Journey of becoming a TensorFlow Certified Developer

A New Decade of Computer Vision

Classification using Long Short Term Memory & GloVe (Global Vectors for Word Representation)

Cat Vs Dog Classification Using Edge Detection And Other Techniques.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Manning Publications

Manning Publications

Follow Manning Publications on Medium for free content and exclusive discounts.

More from Medium

Online prediction using GCP’s Vertex AI

Detection and Normalization of Temporal Expressions in French Text (2) — Label Format and…

Painless Explainability for NLP/Text Models with LIME and ELI5

Deploying Spark NLP for Healthcare: from zero to hero