DeBERTa: State-of-the-art architecture now used in Maigon NDA
We are excited to announce that Maigon NDA, one of our AI applications for contract review, has migrated to the state-of-the-art DeBERTa architecture.
DeBERTa, which stands for Decoding-enhanced BERT with Disentangled Attention, is the latest state-of-the-art architecture developed and released by Microsoft. The architecture significantly improves the BERT and RoBERTa models using two novel techniques: the disentangled attention mechanism and an enhanced mask decoder.
Trained on thousands of contractual clauses, our DeBERTa-based model shows noticeable improvements over the previous iteration (RoBERTa-based).
DeBERTa is currently in top three leader architectures at both GLUE and SuperGLUE natural understanding benchmarks.
Read more about DeBERTa in the model release notes.