Welcome to Apache OpenNLP!

The Apache OpenNLP library is a machine learning based toolkit for the processing of natural language text.

This toolkit is written completely in Java and provides support for common NLP tasks, such as tokenization, sentence segmentation, part-of-speech tagging, named entity extraction, chunking, parsing, coreference resolution, language detection, stopword filtering (with bundled lists for 11 languages) and more!

These tasks are usually required to build more advanced text processing services.

The goal of the OpenNLP project is to be a mature toolkit for the above mentioned tasks.

An additional goal is to provide a large number of pre-built models for a variety of languages, as well as the annotated text resources that those models are derived from.

Presently, OpenNLP includes common classifiers such as Maximum Entropy, Perceptron, Naive Bayes and Support Vector Machines (SVM).

OpenNLP can be used both programmatically through its Java API or from a terminal through its CLI. OpenNLP API can be easily plugged into distributed streaming data pipelines like Apache Flink, Apache NiFi, Apache Spark.

Useful Links

For additional information, visit the OpenNLP Home Page

You can use OpenNLP with any language, demo models are provided here.

The models are fully compatible with the latest release, they can be used for testing or getting started.

[!NOTE]
Please train your own models for all other use cases.

Documentation, including JavaDocs, code usage and command-line interface examples are available here

For recent news, updates and topics, you can:

• join the regular mailing lists,
• follow the project’s social media channel, or
• join the channel (available to people with an @apache.org email address or upon invitation).

Please, also check the community’s OpenNLP questions and answers.

Overview

Currently, the library has different modules:

• opennlp-api : The public API defining core Apache OpenNLP interfaces and abstractions.
• opennlp-runtime : The core classes shared across Apache OpenNLP components.
• opennlp-ml-commons : Common utilities and shared functionality for ML implementations.
• opennlp-ml-maxent : Maximum Entropy (MaxEnt) machine learning implementation.
• opennlp-ml-bayes : Naive Bayes machine learning implementation.
• opennlp-ml-perceptron : Perceptron-based machine learning implementation.
• opennlp-ml-libsvm : Support Vector Machine (SVM) based text classification via zlibsvm.
• opennlp-dl : Apache OpenNLP adapter for ONNX models using the onnxruntime dependency.
• opennlp-dl-gpu : Replaces onnxruntime with the onnxruntime_gpu dependency to support GPU acceleration.
• opennlp-model-resolver : Classes for discovering and loading Apache OpenNLP models from the classpath.
• opennlp-formats : Support for reading and writing various NLP training and data formats.
• opennlp-cli : The command-line tools for training, evaluating, and running models.
• opennlp-tools : The full end-user toolkit with all core components and utilities in its executable form.
• opennlp-morfologik : Extension module providing Morfologik-based dictionary and stemming support.
• opennlp-spellcheck : Extension module providing SymSpell-based, language-agnostic spell correction.
• opennlp-uima : Extension module providing a set of Apache UIMA annotators.
• opennlp-sandbox : Other projects in progress reside in the sandbox.

Getting Started

You can import the core toolkit directly from Maven or Gradle:

Maven

<dependency>
    <groupId>org.apache.opennlp</groupId>
    <artifactId>opennlp-runtime</artifactId>
    <version>${opennlp.version}</version>
</dependency>
<!-- if model support is needed -->
<dependency>
    <groupId>org.apache.opennlp</groupId>
    <artifactId>opennlp-model-resolver</artifactId>
    <version>${opennlp.version}</version>
</dependency>

Note: opennlp-runtime ships with the MaxEnt ML implementation by default. If you need other ML implementations, please add the corresponding dependencies as well.

Gradle

compile group: "org.apache.opennlp", name: "opennlp-runtime", version: "${opennlp.version}"
compile group: "org.apache.opennlp", name: "opennlp-model-resolver", version: "${opennlp.version}"

For more details please check our documentation

Migrating from 2.x to 3.x

The 3.x release line of Apache OpenNLP modularizes the project for better usage as a library and to support future extensibility. The core API remains stable and compatible with 2.x, but the project structure has been reorganized into multiple modules.

That means, that you can continue to use the previous opennlp-tools artifact as a dependency. However, we strongly recommend to switch to the new modular structure and import only the components you need, which will result in a smaller dependency footprint.

Only opennlp-runtime needs to be added as a dependency, and you can add additional modules (e.g. opennlp-ml-maxent, opennlp-model-resolver, etc.) as required by your project. For users of the traditional CLI toolkit, nothing changes with the 3.x release line. CLI usage remains stable as described in the project’s dev manual.

Breaking changes

The list below covers the changes most likely to affect existing code. It is not exhaustive; the release notes and the issue tracker record all changes.

• Whitespace: tokenization, corpus format parsing and feature generation classify whitespace by the Unicode White_Space property. Set -Dopennlp.whitespace.mode=LEGACY to reproduce 2.x tokenization exactly, for example with models trained by 1.x or 2.x.
• AbstractClassPathModelFinder: the protected asRegex and matchesPattern methods were removed. Custom model finders call matchesWildcard(url, "*.bin") instead (OPENNLP-1932).
• BasicContextGenerator(String): the separator is taken literally instead of as a regular expression, and a separator containing a backslash throws IllegalArgumentException. Empty predicates are no longer returned, and the default constructor splits on runs of Unicode whitespace (OPENNLP-1929).
• AD reader: a self-closing markup line such as <s/>, <s /> or <s id="1"/> no longer opens a sentence, paragraph, title, box or text. A tag whose name only starts with s, p, t, caixa or ext, such as <sx>, is no longer read as that tag, and attributes must be separated from the tag name by Unicode whitespace (GH-1447).
• MASC reader: region offsets must be written with ASCII digits only and no sign. A document with a signed or non-ASCII offset is skipped (OPENNLP-1990).
• CoNLL-U reader: a multiword token range whose word lines are missing from the sentence throws InvalidFormatException.
• PlainTextByLineStream.reset() closes the previous reader, so its InputStreamFactory must return a new stream for each pass.
• RealBasicEventStream: a line that contains only an outcome is read as an event without contexts and no longer ends the stream.
• Parser: ParserFactory.create and the chunking and tree insert Parser constructors throw IllegalArgumentException for a beam size below 1 or an advance percentage outside (0, 1] (GH-1476).
• Model resolver: AbstractClassPathModelFinder.findModels returns an unmodifiable set, so the classpath finders and DefaultClassPathModelProvider can be shared between threads (GH-1506).

Thread safety

Starting with 3.0.0, the core *ME classes (POSTaggerME, TokenizerME, SentenceDetectorME, ChunkerME, LemmatizerME, NameFinderME) are thread safe and a single instance can be shared across threads. This eliminates the need to pool or recreate ME instances per thread. The legacy ThreadSafe*ME wrappers from 2.x still work but are now deprecated; existing code does not need to change to upgrade.

Head’s up

The Apache OpenNLP team is planning to change the package namespace from opennlp to org.apache.opennlp in a future release (potentially 4.x). This change will be made to align with standard Java package naming conventions and to avoid potential conflicts with other libraries.

In addition, the Apache OpenNLP team raised the minimal Java version to JDK 21+ for the 3.0.0 release to take advantage of the latest language features and improvements.

Branches and Merging Strategy

To support ongoing development and stable maintenance of Apache OpenNLP, the project follows a dual-branch model:

Branch overview

• main: Development branch for version 3.0 and beyond.

          All feature development and 3.x releases occur here. Minimum Java level: 21.

• opennlp-2.x: Maintains the stable 2.x release line.

          This branch will receive selective updates and patch releases. Minimum Java level: 17.

Workflow summary

• Feature development
  • New features targeting versions 3.0+ are developed on feature branches off main and merged into main.
• Bug fixes and dependency updates
  • Relevant fixes or dependency updates from main may be cherry-picked into opennlp-2.x as needed.
• Releases
  • 3.x releases are made from the main branch.
  • 2.x releases are made from the opennlp-2.x branch.
• Release tags
  • Release tags are applied directly to the appropriate version branch (main for 3.x or opennlp-2.x for 2.x).
  • The presence of a version branch does not affect the tagging or visibility of releases.

Building OpenNLP

For the main branch, at least JDK 21 and Maven 3.9.x are required to build the library.

After cloning the repository go into the destination directory and run:

mvn install

Additional Development Information

• Building and integrating Snowball Stemmer for OpenNLP.

Security

Please do not report security issues through GitHub or the mailing lists. Report them privately to security@apache.org.

See SECURITY.md for the reporting process and for the project’s security model, which describes what Apache OpenNLP treats as trusted and untrusted input.

Contributing

The Apache OpenNLP project is developed by volunteers and is always looking for new contributors to work on all parts of the project. Every contribution is welcome and needed to make it better. A contribution can be anything from a small documentation typo fix to a new component.

If you would like to get involved please follow the instructions here