{"id":1965,"date":"2026-10-28T09:08:59","date_gmt":"2026-10-28T09:08:59","guid":{"rendered":"https:\/\/www.interviewbit.com\/varsity\/blog\/?p=1965"},"modified":"2026-10-28T09:09:03","modified_gmt":"2026-10-28T09:09:03","slug":"smoothing-in-nlp","status":"publish","type":"post","link":"https:\/\/www.interviewbit.com\/varsity\/blog\/smoothing-in-nlp\/","title":{"rendered":"Smoothing in NLP: Laplace, Add-k &#038; Kneser-Ney Explained with Examples"},"content":{"rendered":"\n<div class=\"wp-block-group vlf-insights is-layout-flow wp-block-group-is-layout-flow\">\n<p class=\"vlf-insights-title wp-block-paragraph\">Key Insights<\/p>\n\n\n\n<ul class=\"wp-block-list vlf-insights-list\">\n\n<li>Smoothing in NLP prevents unseen word sequences from receiving zero probability in statistical language models.<\/li>\n\n\n<li>Laplace smoothing adds 1 to every count, while Add-k smoothing adds a configurable value <img decoding=\"async\" src=\"data:image\/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAkAAAASCAYAAACJgPRIAAAABHNCSVQICAgIfAhkiAAAAAFzUkdCAK7OHOkAAACVSURBVCiRzc87DkFhEIbhx7UiCkoKEbEBomYHVqDQKSQam7AGy7ABiUSh0ViDLUgkRzPkOBHa8yWTzOWd\/\/+G\/KqOJKL5C5xg\/21QTOUNHP5BI1win8XX6zRcwBntqFeYR185ml08cMcGx4gPvZ5PcMrYeBdDLNHBGIM0VApwix2u6KOCKlq4QS+81GJxGkcssp7ypCeKsBZ+Y6FPgQAAAABJRU5ErkJggg==\" alt=\"\">, allowing more flexibility in probability estimation.<\/li>\n\n\n<li>Both methods use the vocabulary size to adjust the denominator and distribute probability mass across possible words.<\/li>\n\n\n<li>Good-Turing, interpolation and backoff offer alternative approaches to estimating probabilities for unseen or rare n-grams.<\/li>\n\n\n<li>Kneser-Ney smoothing combines discounted counts with continuation probabilities, making it more informative than simply adding a constant to counts.<\/li>\n\n\n<li>Smoothing affects language-model evaluation by changing probability estimates and perplexity; the best method depends on the model and evaluation data.<\/li>\n\n\n<li>Laplace smoothing can also be applied to Naive Bayes classifiers to prevent zero probabilities when a feature is absent from a class.<\/li>\n\n<\/ul>\n<\/div>\n\n\n\n<p class=\"wp-block-paragraph\">Smoothing in NLP is a technique used in statistical n-gram language models to assign non-zero probabilities to word sequences that were not observed in training data. Laplace and Add-k smoothing adjust counts directly, while Kneser-Ney uses discounted counts and continuation probabilities to produce more informative estimates.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">But why does this matter? When a language model encounters an unseen word sequence, its raw probability estimate may become zero, affecting predictions and evaluation. This guide explains the zero-probability problem, compares key smoothing techniques in NLP using a shared bigram example, and demonstrates a basic Python implementation.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1774\" height=\"887\" src=\"https:\/\/www.interviewbit.com\/varsity\/blog\/wp-content\/uploads\/2026\/10\/overview-of-nlp-smoothing-techniques-showing-how-laplace-add.png\" alt=\"Overview of NLP smoothing techniques showing how Laplace, Add-k and Kneser-Ney address zero-probability estimates in language models.\" class=\"wp-image-1967\" srcset=\"https:\/\/www.interviewbit.com\/varsity\/blog\/wp-content\/uploads\/2026\/10\/overview-of-nlp-smoothing-techniques-showing-how-laplace-add.png 1774w, https:\/\/www.interviewbit.com\/varsity\/blog\/wp-content\/uploads\/2026\/10\/overview-of-nlp-smoothing-techniques-showing-how-laplace-add-300x150.png 300w, https:\/\/www.interviewbit.com\/varsity\/blog\/wp-content\/uploads\/2026\/10\/overview-of-nlp-smoothing-techniques-showing-how-laplace-add-1024x512.png 1024w, https:\/\/www.interviewbit.com\/varsity\/blog\/wp-content\/uploads\/2026\/10\/overview-of-nlp-smoothing-techniques-showing-how-laplace-add-768x384.png 768w, https:\/\/www.interviewbit.com\/varsity\/blog\/wp-content\/uploads\/2026\/10\/overview-of-nlp-smoothing-techniques-showing-how-laplace-add-1536x768.png 1536w\" sizes=\"auto, (max-width: 1774px) 100vw, 1774px\" \/><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\">Why is Smoothing Needed in NLP?<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Smoothing prevents unseen n-grams from receiving zero probability in a statistical language model.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">The Zero-Probability Problem in N-Gram Models<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">An <em>n-gram<\/em> is a sequence of <em>n<\/em> consecutive tokens: a unigram is one token, a bigram is two, and a trigram is three. N-gram language models estimate the probability of a word based on its preceding context.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">For a bigram, the maximum-likelihood estimate is:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">P(w&#7522; | w&#7522;&#8331;&#8321;) = C(w&#7522;&#8331;&#8321;, w&#7522;) \/ C(w&#7522;&#8331;&#8321;)<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Here, C(w&#7522;&#8331;&#8321;, w&#7522;) is the bigram count, and C(w&#7522;&#8331;&#8321;) is the context count.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">If &ldquo;like enjoy&rdquo; never appears in the training corpus, its estimated probability is zero. Since sentence probabilities are calculated by multiplying conditional probabilities, one unseen bigram can make the entire sentence probability zero. This is the zero probability problem in NLP, often caused by data sparsity.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">How Smoothing Helps<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Smoothing adjusts probability estimates to account for unseen events while maintaining a normalised probability distribution. It is useful in count-based language models for applications such as speech recognition, text prediction and spelling correction.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">What is Laplace Smoothing in NLP?<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Laplace smoothing, also called Add-One smoothing, adds one to every possible n-gram count. This prevents zero counts from producing zero probabilities, although it can assign excessive probability to rare or unseen events.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Laplace Smoothing Formula<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">For bigrams, the Laplace smoothing formula is:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">P&#8343;&#8336;&#8346;&#8343;&#8336;c&#8337;(w&#7522; | w&#7522;&#8331;&#8321;) = [C(w&#7522;&#8331;&#8321;, w&#7522;) + 1] \/ [C(w&#7522;&#8331;&#8321;) + V]<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Here, (V) is the vocabulary size: the number of possible next tokens. The denominator increases by (V) because one is added to each possible next-token count.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Worked Bigram Example: Calculating Laplace Probabilities<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Consider this illustrative corpus:<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>&ldquo;I like NLP&rdquo;<\/li><li>&ldquo;I like AI&rdquo;<\/li><li>&ldquo;I enjoy AI&rdquo;<\/li><\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Assume consistent sentence-boundary handling and a prediction vocabulary of (V=6). The context &ldquo;like&rdquo; occurs twice, followed by &ldquo;NLP&rdquo; once and &ldquo;AI&rdquo; once.<\/p>\n\n\n\n<figure class=\"wp-block-table\"><table class=\"has-fixed-layout\"><thead><tr><th>Next word after &ldquo;like&rdquo;<\/th><th>Count<\/th><th>Laplace calculation<\/th><th>Probability<\/th><\/tr><\/thead><tbody><tr><td>NLP<\/td><td>1<\/td><td>((1+1)\/(2+6))<\/td><td>0.25<\/td><\/tr><tr><td>AI<\/td><td>1<\/td><td>((1+1)\/(2+6))<\/td><td>0.25<\/td><\/tr><tr><td>enjoy<\/td><td>0<\/td><td>((0+1)\/(2+6))<\/td><td>0.125<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">Although &ldquo;enjoy&rdquo; never follows &ldquo;like&rdquo; in the corpus, it receives a non-zero probability. These are three of the six possible next tokens; the other three also receive probability mass. The example uses a fixed vocabulary to keep the arithmetic simple.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">What is Add-k Smoothing, and How is it Different from Laplace Smoothing?<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Add-k smoothing adds a constant (k) to every possible n-gram count rather than always adding one. Laplace smoothing is the special case where (k=1); smaller positive values make the adjustment less aggressive.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Add-k Smoothing Formula and Example<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">The Add-k smoothing formula is:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">P&#8342;(w&#7522; | w&#7522;&#8331;&#8321;) = [C(w&#7522;&#8331;&#8321;, w&#7522;) + k] \/ [C(w&#7522;&#8331;&#8321;) + kV]<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Using the same example, let (k=0.5) and (V=6). The denominator is (2+(0.5\\times6)=5).<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>For &ldquo;NLP&rdquo;: ((1+0.5)\/5=0.30)<\/li><li>For unseen &ldquo;enjoy&rdquo;: ((0+0.5)\/5=0.10)<\/li><\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">The unseen word receives a probability of 0.10 instead of 0.125 under Laplace smoothing. A smaller (k) reduces the count adjustment, but it does not automatically improve model performance.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Laplace vs Add-k Smoothing<\/h3>\n\n\n\n<figure class=\"wp-block-table\"><table class=\"has-fixed-layout\"><thead><tr><th>Aspect<\/th><th>Laplace<\/th><th>Add-k<\/th><\/tr><\/thead><tbody><tr><td>Count adjustment<\/td><td>Adds 1<\/td><td>Adds chosen (k)<\/td><\/tr><tr><td>Relationship<\/td><td>Special case of Add-k<\/td><td>General additive method<\/td><\/tr><tr><td>Main benefit<\/td><td>Simple baseline<\/td><td>Adjustable smoothing strength<\/td><\/tr><tr><td>Main limitation<\/td><td>Can over-smooth<\/td><td>Requires choosing (k)<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\">What are Good-Turing, Interpolation, and Backoff Smoothing?<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">These smoothing techniques in NLP estimate unseen events using count patterns or information from lower-order n-grams.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Good-Turing Smoothing<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Good-Turing smoothing uses frequency-of-frequency counts&mdash;the number of n-grams observed once, twice and so on&mdash;to estimate how much probability should be reserved for unseen events. It adjusts estimates based on the distribution of observed counts rather than simply adding a fixed constant.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Interpolation and Backoff<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Interpolation smoothing in NLP combines estimates from multiple n-gram orders, such as unigram, bigram and trigram probabilities, using weights.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Backoff smoothing uses a lower-order estimate when a higher-order n-gram is unseen or lacks sufficient evidence. For example, if a trigram is missing, the model may fall back to its bigram estimate. The key distinction in smoothing vs backoff is that interpolation combines estimates, while backoff conditionally falls back to a lower-order model.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">What is Kneser-Ney Smoothing?<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Kneser-Ney smoothing discounts observed n-gram counts and redistributes some probability mass using lower-order continuation probabilities. Instead of relying only on a word&#39;s overall frequency, it considers how many distinct contexts the word follows.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Kneser-Ney Smoothing Intuition<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Discounting reduces observed counts slightly and reallocates the removed probability mass to other possibilities. Continuation probability reflects how many distinct preceding contexts a word appears after.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Imagine one word frequently follows a single common word, while another follows many different words. The second may be a more useful general continuation, even if its total count is lower. Kneser-Ney captures this distinction.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Modified Kneser-Ney uses different discounts for different count ranges. It is a strong classical method for n-gram language modelling, but no smoothing method is universally best across all datasets and evaluation settings.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">How can you Implement Laplace Smoothing in Python?<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">A basic Laplace smoother adds one to the observed count and the vocabulary size to the context count. This illustrates smoothing in NLP Python without requiring a full language-model implementation.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Simple Python Example<\/h3>\n\n\n\n<pre class=\"wp-block-code\"><code>def laplace_smoothing(count, context_count, vocab_size):\n    return (count + 1) \/ (context_count + vocab_size)\n# \"NLP\" appears once after \"like\"\np_seen = laplace_smoothing(1, 2, 6)\n# \"enjoy\" never appears after \"like\"\np_unseen = laplace_smoothing(0, 2, 6)\nprint(p_seen)    # 0.25\nprint(p_unseen)  # 0.125<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">The inputs are the bigram count, context count and vocabulary size. The vocabulary must be defined consistently. This function calculates one probability; a complete language model must also handle tokenisation, sentence boundaries and counts.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">How is Laplace Smoothing Used in Naive Bayes?<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">In text classification, Laplace smoothing in Naive Bayes prevents a word absent from a class&#39;s training documents from making its likelihood zero.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The general likelihood formula is:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">P(w | c) = [C(w, c) + &alpha;] \/ [N&#42752; + &alpha;V]<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Here, C(w, c) is the count of word w in context or class c, N&#42752; is the total count of observations in context or class c, &alpha; is the smoothing parameter, and V is the vocabulary size. In scikit-learn&#39;s MultinomialNB, the &alpha; parameter provides additive smoothing: (&alpha;=1) corresponds to Laplace smoothing, while smaller positive values correspond to Lidstone smoothing.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>How Does Smoothing Affect Perplexity?<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Language model perplexity measures how well a model predicts a test sequence. Smoothing changes probability estimates and can therefore affect perplexity, but it does not guarantee improvement. A zero-probability test sequence can make the usual perplexity calculation infinite or undefined.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Compare smoothing methods on held-out data rather than choosing one based only on formula simplicity.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Conclusion<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Smoothing helps statistical language models handle unseen word sequences. Laplace provides a simple baseline, Add-k makes the adjustment tunable, and Kneser-Ney uses discounted counts and continuation probabilities for more context-aware estimates. Good-Turing, interpolation and backoff offer other ways to address sparse counts. The most suitable method depends on the data and should be evaluated on held-out examples.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Want to build a stronger foundation in NLP and understand how statistical language models connect with modern Generative AI? Explore the <a href=\"https:\/\/www.interviewbit.com\/varsity\/iit-delhi\/generative-ai\"><strong>Generative AI programme by IIT Delhi<\/strong><\/a>, delivered on Varsity by InterviewBit to deepen your understanding of the techniques behind today&rsquo;s AI systems.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Frequently Asked Questions<\/h2>\n\n\n\n<h3 class=\"wp-block-heading has-medium-font-size\">Does smoothing create new word sequences in the training corpus?<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">No. Smoothing changes the probability estimates assigned to possible events, including sequences not observed in the training data.<\/p>\n\n\n\n<h3 class=\"wp-block-heading has-medium-font-size\">Can smoothing assign probability to a word outside the vocabulary?<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Not by itself. The model needs an explicit strategy, such as mapping unknown words to an &lt;UNK&gt; token.<\/p>\n\n\n\n<h3 class=\"wp-block-heading has-medium-font-size\">Does every smoothing method preserve a valid probability distribution?<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">A correctly formulated and normalised method should assign probabilities that sum to one over the defined outcomes. Counts and vocabulary definitions must remain consistent.<\/p>\n\n\n\n<h3 class=\"wp-block-heading has-medium-font-size\">What happens if the smoothing parameter is too large?<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Excessive additive smoothing can flatten the probability distribution, making observed and unseen events less distinguishable.<\/p>\n\n\n\n<h3 class=\"wp-block-heading has-medium-font-size\">Is Add-k smoothing the same as discounting?<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">No. Add-k increases counts by a constant, while discounting methods reduce observed counts and redistribute probability mass.<\/p>\n\n\n\n<h3 class=\"wp-block-heading has-medium-font-size\">Can smoothing be applied to trigrams as well as bigrams?<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Yes. The same general principles apply to higher-order n-grams, where sparsity often becomes more severe as context length increases.<\/p>\n\n\n\n<h3 class=\"wp-block-heading has-medium-font-size\">How should I choose the value of k in Add-k smoothing?<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Compare candidate values on held-out validation data. The best value depends on the corpus and task.<\/p>\n\n\n\n<h3 class=\"wp-block-heading has-medium-font-size\">Why can a frequent word still have a low continuation probability?<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Continuation probability depends on the number of distinct preceding contexts, not simply the word&#39;s total frequency.<\/p>\n\n\n\n<h3 class=\"wp-block-heading has-medium-font-size\">Is Kneser-Ney smoothing used in modern large language models?<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Kneser-Ney is a classical technique for count-based n-gram models. Modern neural language models generally use different training and probability-estimation approaches.<\/p>\n\n\n\n<h3 class=\"wp-block-heading has-medium-font-size\">Should I always use smoothing when training a language model?<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Smoothing is important for many sparse count-based models, but the method and its necessity depend on the model family, vocabulary handling and evaluation setup.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Resources<\/h2>\n\n\n\n<ul class=\"wp-block-list\"><li><a href=\"https:\/\/www.nltk.org\/api\/nltk.lm.smoothing.html\">Laplace Smoothing and Kneser-Ney Smoothing &mdash; NLTK Documentation<\/a><\/li><li><a href=\"https:\/\/www.nltk.org\/api\/nltk.lm.html\">Language Modelling with NLTK<\/a><\/li><li><a href=\"https:\/\/scikit-learn.org\/stable\/modules\/generated\/sklearn.naive_bayes.MultinomialNB.html\">Multinomial Naive Bayes &mdash; scikit-learn Documentation<\/a><\/li><li><a href=\"https:\/\/scikit-learn.org\/stable\/modules\/naive_bayes.html\">Naive Bayes &mdash; scikit-learn User Guide<\/a><\/li><li><a href=\"https:\/\/www.aclweb.org\/anthology\/P08-2036.pdf\">Smoothing a Tera-Word Language Model<\/a><\/li><\/ul>\n","protected":false},"excerpt":{"rendered":"<p>Key Insights Smoothing in NLP is a technique used in statistical n-gram language models to assign non-zero probabilities to word sequences that were not observed in training data. Laplace and Add-k smoothing adjust counts directly, while Kneser-Ney uses discounted counts and continuation probabilities to produce more informative estimates. But why does this matter? When a [&hellip;]<\/p>\n","protected":false},"author":7,"featured_media":1987,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[7],"tags":[],"class_list":["post-1965","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-generative-ai"],"blocksy_meta":{"styles_descriptor":{"styles":{"desktop":"","tablet":"","mobile":""},"google_fonts":[],"version":8}},"acf":{"reviewed_by":""},"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v28.8 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Smoothing in NLP: Laplace, Add-k &amp; Kneser-Ney Explained with Examples<\/title>\n<meta name=\"description\" content=\"Understand smoothing in NLP, including Laplace, Add-k and Kneser-Ney smoothing, with worked bigram examples, formulas, Python implementation and perplexity.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.interviewbit.com\/varsity\/blog\/smoothing-in-nlp\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Smoothing in NLP: Laplace, Add-k &amp; Kneser-Ney Explained with Examples\" \/>\n<meta property=\"og:description\" content=\"Understand smoothing in NLP, including Laplace, Add-k and Kneser-Ney smoothing, with worked bigram examples, formulas, Python implementation and perplexity.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.interviewbit.com\/varsity\/blog\/smoothing-in-nlp\/\" \/>\n<meta property=\"og:site_name\" content=\"Varsity Blog\" \/>\n<meta property=\"article:published_time\" content=\"2026-10-28T09:08:59+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2026-10-28T09:09:03+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.interviewbit.com\/varsity\/blog\/wp-content\/uploads\/2026\/10\/smoothing-in-nlp-.webp\" \/>\n\t<meta property=\"og:image:width\" content=\"1010\" \/>\n\t<meta property=\"og:image:height\" content=\"673\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/webp\" \/>\n<meta name=\"author\" content=\"Varsity on Behalf of CEP IIT Delhi\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Varsity on Behalf of CEP IIT Delhi\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"8 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/www.interviewbit.com\\\/varsity\\\/blog\\\/smoothing-in-nlp\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.interviewbit.com\\\/varsity\\\/blog\\\/smoothing-in-nlp\\\/\"},\"author\":{\"name\":\"Varsity on Behalf of CEP IIT Delhi\",\"@id\":\"https:\\\/\\\/www.interviewbit.com\\\/varsity\\\/blog\\\/#\\\/schema\\\/person\\\/7b5db9a94eddee529cd35968692d9c29\"},\"headline\":\"Smoothing in NLP: Laplace, Add-k &#038; Kneser-Ney Explained with Examples\",\"datePublished\":\"2026-10-28T09:08:59+00:00\",\"dateModified\":\"2026-10-28T09:09:03+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/www.interviewbit.com\\\/varsity\\\/blog\\\/smoothing-in-nlp\\\/\"},\"wordCount\":1692,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\\\/\\\/www.interviewbit.com\\\/varsity\\\/blog\\\/#organization\"},\"image\":{\"@id\":\"https:\\\/\\\/www.interviewbit.com\\\/varsity\\\/blog\\\/smoothing-in-nlp\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/www.interviewbit.com\\\/varsity\\\/blog\\\/wp-content\\\/uploads\\\/2026\\\/10\\\/smoothing-in-nlp-.webp\",\"articleSection\":[\"Generative AI\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\\\/\\\/www.interviewbit.com\\\/varsity\\\/blog\\\/smoothing-in-nlp\\\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/www.interviewbit.com\\\/varsity\\\/blog\\\/smoothing-in-nlp\\\/\",\"url\":\"https:\\\/\\\/www.interviewbit.com\\\/varsity\\\/blog\\\/smoothing-in-nlp\\\/\",\"name\":\"Smoothing in NLP: Laplace, Add-k & Kneser-Ney Explained with Examples\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.interviewbit.com\\\/varsity\\\/blog\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/www.interviewbit.com\\\/varsity\\\/blog\\\/smoothing-in-nlp\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/www.interviewbit.com\\\/varsity\\\/blog\\\/smoothing-in-nlp\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/www.interviewbit.com\\\/varsity\\\/blog\\\/wp-content\\\/uploads\\\/2026\\\/10\\\/smoothing-in-nlp-.webp\",\"datePublished\":\"2026-10-28T09:08:59+00:00\",\"dateModified\":\"2026-10-28T09:09:03+00:00\",\"description\":\"Understand smoothing in NLP, including Laplace, Add-k and Kneser-Ney smoothing, with worked bigram examples, formulas, Python implementation and perplexity.\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/www.interviewbit.com\\\/varsity\\\/blog\\\/smoothing-in-nlp\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/www.interviewbit.com\\\/varsity\\\/blog\\\/smoothing-in-nlp\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/www.interviewbit.com\\\/varsity\\\/blog\\\/smoothing-in-nlp\\\/#primaryimage\",\"url\":\"https:\\\/\\\/www.interviewbit.com\\\/varsity\\\/blog\\\/wp-content\\\/uploads\\\/2026\\\/10\\\/smoothing-in-nlp-.webp\",\"contentUrl\":\"https:\\\/\\\/www.interviewbit.com\\\/varsity\\\/blog\\\/wp-content\\\/uploads\\\/2026\\\/10\\\/smoothing-in-nlp-.webp\",\"width\":1010,\"height\":673,\"caption\":\"smoothing-in-nlp-\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/www.interviewbit.com\\\/varsity\\\/blog\\\/smoothing-in-nlp\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/www.interviewbit.com\\\/varsity\\\/blog\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Smoothing in NLP: Laplace, Add-k &#038; Kneser-Ney Explained with Examples\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/www.interviewbit.com\\\/varsity\\\/blog\\\/#website\",\"url\":\"https:\\\/\\\/www.interviewbit.com\\\/varsity\\\/blog\\\/\",\"name\":\"Varsity Blog\",\"description\":\"\",\"publisher\":{\"@id\":\"https:\\\/\\\/www.interviewbit.com\\\/varsity\\\/blog\\\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/www.interviewbit.com\\\/varsity\\\/blog\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/www.interviewbit.com\\\/varsity\\\/blog\\\/#organization\",\"name\":\"Varsity Blog\",\"url\":\"https:\\\/\\\/www.interviewbit.com\\\/varsity\\\/blog\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/www.interviewbit.com\\\/varsity\\\/blog\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"https:\\\/\\\/www.interviewbit.com\\\/varsity\\\/blog\\\/wp-content\\\/uploads\\\/2026\\\/08\\\/varsity-logo.png\",\"contentUrl\":\"https:\\\/\\\/www.interviewbit.com\\\/varsity\\\/blog\\\/wp-content\\\/uploads\\\/2026\\\/08\\\/varsity-logo.png\",\"width\":275,\"height\":64,\"caption\":\"Varsity Blog\"},\"image\":{\"@id\":\"https:\\\/\\\/www.interviewbit.com\\\/varsity\\\/blog\\\/#\\\/schema\\\/logo\\\/image\\\/\"},\"sameAs\":[\"https:\\\/\\\/www.linkedin.com\\\/company\\\/varsity-by-interviewbit\\\/\"]},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/www.interviewbit.com\\\/varsity\\\/blog\\\/#\\\/schema\\\/person\\\/7b5db9a94eddee529cd35968692d9c29\",\"name\":\"Varsity on Behalf of CEP IIT Delhi\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/bdb12370d043ff980b2b8f1ccb67f5a1f0e333aaca46cc35358b1af8b1d98334?s=96&d=mm&r=g\",\"url\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/bdb12370d043ff980b2b8f1ccb67f5a1f0e333aaca46cc35358b1af8b1d98334?s=96&d=mm&r=g\",\"contentUrl\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/bdb12370d043ff980b2b8f1ccb67f5a1f0e333aaca46cc35358b1af8b1d98334?s=96&d=mm&r=g\",\"caption\":\"Varsity on Behalf of CEP IIT Delhi\"},\"description\":\"Varsity by InterviewBit, in collaboration with CEP IIT Delhi, creates industry-relevant learning programmes designed to help learners build practical, in-demand skills. Through this author profile, we publish articles that complement our courses covering curriculum-aligned topics, foundational concepts, emerging trends, and advanced insights. Our goal is to help learners deepen their understanding beyond the classroom and apply their knowledge confidently in real-world contexts.\",\"url\":\"https:\\\/\\\/www.interviewbit.com\\\/varsity\\\/blog\\\/author\\\/varsity-on-behalf-of-cep-iit-delhi\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Smoothing in NLP: Laplace, Add-k & Kneser-Ney Explained with Examples","description":"Understand smoothing in NLP, including Laplace, Add-k and Kneser-Ney smoothing, with worked bigram examples, formulas, Python implementation and perplexity.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.interviewbit.com\/varsity\/blog\/smoothing-in-nlp\/","og_locale":"en_US","og_type":"article","og_title":"Smoothing in NLP: Laplace, Add-k & Kneser-Ney Explained with Examples","og_description":"Understand smoothing in NLP, including Laplace, Add-k and Kneser-Ney smoothing, with worked bigram examples, formulas, Python implementation and perplexity.","og_url":"https:\/\/www.interviewbit.com\/varsity\/blog\/smoothing-in-nlp\/","og_site_name":"Varsity Blog","article_published_time":"2026-10-28T09:08:59+00:00","article_modified_time":"2026-10-28T09:09:03+00:00","og_image":[{"width":1010,"height":673,"url":"https:\/\/www.interviewbit.com\/varsity\/blog\/wp-content\/uploads\/2026\/10\/smoothing-in-nlp-.webp","type":"image\/webp"}],"author":"Varsity on Behalf of CEP IIT Delhi","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Varsity on Behalf of CEP IIT Delhi","Est. reading time":"8 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.interviewbit.com\/varsity\/blog\/smoothing-in-nlp\/#article","isPartOf":{"@id":"https:\/\/www.interviewbit.com\/varsity\/blog\/smoothing-in-nlp\/"},"author":{"name":"Varsity on Behalf of CEP IIT Delhi","@id":"https:\/\/www.interviewbit.com\/varsity\/blog\/#\/schema\/person\/7b5db9a94eddee529cd35968692d9c29"},"headline":"Smoothing in NLP: Laplace, Add-k &#038; Kneser-Ney Explained with Examples","datePublished":"2026-10-28T09:08:59+00:00","dateModified":"2026-10-28T09:09:03+00:00","mainEntityOfPage":{"@id":"https:\/\/www.interviewbit.com\/varsity\/blog\/smoothing-in-nlp\/"},"wordCount":1692,"commentCount":0,"publisher":{"@id":"https:\/\/www.interviewbit.com\/varsity\/blog\/#organization"},"image":{"@id":"https:\/\/www.interviewbit.com\/varsity\/blog\/smoothing-in-nlp\/#primaryimage"},"thumbnailUrl":"https:\/\/www.interviewbit.com\/varsity\/blog\/wp-content\/uploads\/2026\/10\/smoothing-in-nlp-.webp","articleSection":["Generative AI"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.interviewbit.com\/varsity\/blog\/smoothing-in-nlp\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.interviewbit.com\/varsity\/blog\/smoothing-in-nlp\/","url":"https:\/\/www.interviewbit.com\/varsity\/blog\/smoothing-in-nlp\/","name":"Smoothing in NLP: Laplace, Add-k & Kneser-Ney Explained with Examples","isPartOf":{"@id":"https:\/\/www.interviewbit.com\/varsity\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.interviewbit.com\/varsity\/blog\/smoothing-in-nlp\/#primaryimage"},"image":{"@id":"https:\/\/www.interviewbit.com\/varsity\/blog\/smoothing-in-nlp\/#primaryimage"},"thumbnailUrl":"https:\/\/www.interviewbit.com\/varsity\/blog\/wp-content\/uploads\/2026\/10\/smoothing-in-nlp-.webp","datePublished":"2026-10-28T09:08:59+00:00","dateModified":"2026-10-28T09:09:03+00:00","description":"Understand smoothing in NLP, including Laplace, Add-k and Kneser-Ney smoothing, with worked bigram examples, formulas, Python implementation and perplexity.","breadcrumb":{"@id":"https:\/\/www.interviewbit.com\/varsity\/blog\/smoothing-in-nlp\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.interviewbit.com\/varsity\/blog\/smoothing-in-nlp\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.interviewbit.com\/varsity\/blog\/smoothing-in-nlp\/#primaryimage","url":"https:\/\/www.interviewbit.com\/varsity\/blog\/wp-content\/uploads\/2026\/10\/smoothing-in-nlp-.webp","contentUrl":"https:\/\/www.interviewbit.com\/varsity\/blog\/wp-content\/uploads\/2026\/10\/smoothing-in-nlp-.webp","width":1010,"height":673,"caption":"smoothing-in-nlp-"},{"@type":"BreadcrumbList","@id":"https:\/\/www.interviewbit.com\/varsity\/blog\/smoothing-in-nlp\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.interviewbit.com\/varsity\/blog\/"},{"@type":"ListItem","position":2,"name":"Smoothing in NLP: Laplace, Add-k &#038; Kneser-Ney Explained with Examples"}]},{"@type":"WebSite","@id":"https:\/\/www.interviewbit.com\/varsity\/blog\/#website","url":"https:\/\/www.interviewbit.com\/varsity\/blog\/","name":"Varsity Blog","description":"","publisher":{"@id":"https:\/\/www.interviewbit.com\/varsity\/blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.interviewbit.com\/varsity\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.interviewbit.com\/varsity\/blog\/#organization","name":"Varsity Blog","url":"https:\/\/www.interviewbit.com\/varsity\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.interviewbit.com\/varsity\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/www.interviewbit.com\/varsity\/blog\/wp-content\/uploads\/2026\/08\/varsity-logo.png","contentUrl":"https:\/\/www.interviewbit.com\/varsity\/blog\/wp-content\/uploads\/2026\/08\/varsity-logo.png","width":275,"height":64,"caption":"Varsity Blog"},"image":{"@id":"https:\/\/www.interviewbit.com\/varsity\/blog\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.linkedin.com\/company\/varsity-by-interviewbit\/"]},{"@type":"Person","@id":"https:\/\/www.interviewbit.com\/varsity\/blog\/#\/schema\/person\/7b5db9a94eddee529cd35968692d9c29","name":"Varsity on Behalf of CEP IIT Delhi","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/bdb12370d043ff980b2b8f1ccb67f5a1f0e333aaca46cc35358b1af8b1d98334?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/bdb12370d043ff980b2b8f1ccb67f5a1f0e333aaca46cc35358b1af8b1d98334?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/bdb12370d043ff980b2b8f1ccb67f5a1f0e333aaca46cc35358b1af8b1d98334?s=96&d=mm&r=g","caption":"Varsity on Behalf of CEP IIT Delhi"},"description":"Varsity by InterviewBit, in collaboration with CEP IIT Delhi, creates industry-relevant learning programmes designed to help learners build practical, in-demand skills. Through this author profile, we publish articles that complement our courses covering curriculum-aligned topics, foundational concepts, emerging trends, and advanced insights. Our goal is to help learners deepen their understanding beyond the classroom and apply their knowledge confidently in real-world contexts.","url":"https:\/\/www.interviewbit.com\/varsity\/blog\/author\/varsity-on-behalf-of-cep-iit-delhi\/"}]}},"_links":{"self":[{"href":"https:\/\/www.interviewbit.com\/varsity\/blog\/wp-json\/wp\/v2\/posts\/1965","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.interviewbit.com\/varsity\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.interviewbit.com\/varsity\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.interviewbit.com\/varsity\/blog\/wp-json\/wp\/v2\/users\/7"}],"replies":[{"embeddable":true,"href":"https:\/\/www.interviewbit.com\/varsity\/blog\/wp-json\/wp\/v2\/comments?post=1965"}],"version-history":[{"count":2,"href":"https:\/\/www.interviewbit.com\/varsity\/blog\/wp-json\/wp\/v2\/posts\/1965\/revisions"}],"predecessor-version":[{"id":1968,"href":"https:\/\/www.interviewbit.com\/varsity\/blog\/wp-json\/wp\/v2\/posts\/1965\/revisions\/1968"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.interviewbit.com\/varsity\/blog\/wp-json\/wp\/v2\/media\/1987"}],"wp:attachment":[{"href":"https:\/\/www.interviewbit.com\/varsity\/blog\/wp-json\/wp\/v2\/media?parent=1965"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.interviewbit.com\/varsity\/blog\/wp-json\/wp\/v2\/categories?post=1965"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.interviewbit.com\/varsity\/blog\/wp-json\/wp\/v2\/tags?post=1965"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}