Improving Embeddings for High-Accuracy Transformer-Based Address Matching Using a Multiple in-Batch Negatives Loss
See where this sits in the topic map →Summary AI-generated
- TL;DR
- Accurate address matching is crucial for parcel delivery, as errors lead to economic and environmental costs as well as reputational damage.
- Problem
- Inaccurate parcel delivery caused by poorly matched addresses creates significant economic, environmental, and reputational costs for logistics companies.
- Method
- The paper proposes a bi-encoder deep learning model to create meaningful embeddings of Portuguese postal addresses, retrieving matches from a normalized database.
- Results
- Tested on real-world Portuguese address data, the model achieved a mapping accuracy exceeding 99.60% at the door level. The research also found that training the transformer from scratch yielded better results than using a pre-trained model.
- Contributions
- The study demonstrates that training a transformer model from scratch with a multiple negatives ranking loss provides optimal weight initialization and high accuracy for address matching.
- Limitations
- Not specified in the abstract.
- Takeaways
- Implementing this high-accuracy address matching system in real-world parcel distribution is expected to bring significant efficiency gains and is currently under evaluation.
- Applications
- Postal services and companies responsible for parcel processing and delivery.
- Topics
- Address matching, deep learning, bi-encoder, embeddings, transformers
- For industry
- Postal services, logistics, and parcel delivery
- Why it matters
- Improves delivery efficiency, reducing economic and environmental costs associated with misdelivered parcels.
Abstract
Address matching is a crucial activity for post offices and companies responsible for parcel processing and delivery. Inaccurate delivery of parcels can significantly impact the reputation of these companies and result in considerable economic and environmental costs. This paper proposes a deep learning model that aims to increase efficiency on the address matching task for portuguese addresses. The model consists on a bi-encoder, trained to create meaningful embeddings of portuguese postal addresses, which is then used to retrieve from a normalized database the matches of the target unnormalized addresses. We argue that a good initialization of the bi-encoder weights is a crucial step for achieving optimal performance and we support our hypothesis by showing that training a transformer from scratch leads to better results, when compared with using a pre-trained model. We also evaluate the bi-encoder's performance when using a standard contrastive loss, where we carefully select the negative samples, versus using a multiple negatives ranking loss, where we use larger batch sizes with multiple random in-batch negatives. The model, trained from scratch with the multiple negatives ranking loss, was tested with data retrieved from a real-life scenario of portuguese addresses and exhibited a very high mapping accuracy, exceeding 99.60% at the door level. The implementation of this system in a real context of parcel deliveries is expected to result in significant efficiency gains in the distribution process. Such an implementation is currently under evaluation.