Products Roadmap Clients News About

DeBERTa: State-of-the-art architecture now used in Maigon NDA Apr 03, 2021

All news

We are excited to announce that Maigon NDA, one of our AI applications for contract review, has migrated to the state-of-the-art DeBERTa architecture.

DeBERTa, which stands for Decoding-enhanced BERT with Disentangled Attention, is the latest state-of-the-art architecture developed and released by Microsoft. The architecture significantly improves the BERT and RoBERTa models using two novel techniques: the disentangled attention mechanism and an enhanced mask decoder.

Trained on thousands of contractual clauses, our DeBERTa-based model shows noticeable improvements over the previous iteration (RoBERTa-based).

DeBERTa is currently in top three leader architectures at both GLUE and SuperGLUE natural understanding benchmarks.

Read more about DeBERTa in the model release notes.