particular.net Open in urlscan Pro
104.40.215.219  Public Scan

Form analysis
2 forms found in the DOM

POST

<form id="blog-subscription-form" method="post">
  <div class="message"><strong>Don't miss a thing. </strong>Sign up today and we'll send you an email when new posts come out.</div>
  <div class="success">Thank you for subscribing. We'll be in touch soon.</div>
  <div class="input-group"> <input class="email-address" name="request-email" placeholder="Email address"> &nbsp; <button class="submit">Get notified</button> </div>
  <div class="registration-disclaimer">We collect and use this information in accordance with our <a href="/gdpr-privacy" target="_blank">privacy policy</a>.</div>
</form>

<form class="needs-validation" novalidate="">
  <div class="hide alert alert-danger" role="alert"></div>
  <p>Required fields are marked with <abbr class="required" title="required">*</abbr></p>
  <div class="form-row">
    <div class="form-group col-12 col-sm-6"> <input class="form-control" name="FirstName" id="wtu-first-name" required=""> <label for="wtu-first-name"> <span>First name <abbr class="required" title="required">*</abbr></span>
        <div class="invalid-feedback">Missing</div>
      </label> </div>
    <div class="form-group col-12 col-sm-6"> <input class="form-control" name="LastName" id="wtu-last-name" required=""> <label for="wtu-last-name"> <span>Last name <abbr class="required" title="required">*</abbr></span>
        <div class="invalid-feedback">Missing</div>
      </label> </div>
  </div>
  <div class="form-group"> <input class="form-control" name="Company" id="wtu-company"> <label for="wtu-company"> <span>Company name</span> </label> </div>
  <div class="form-group"> <input class="form-control" name="Email" id="wtu-email-address" required="" pattern="^[a-z0-9._%+-]+@[a-z0-9.-]+\.[a-z]{2,}$" type="email"> <label for="wtu-email-address"> <span>Email address <abbr class="required"
          title="required">*</abbr></span>
      <div class="invalid-feedback">Invalid</div>
    </label> </div>
  <div class="form-group"> <input class="form-control" name="Phone" id="wtu-phone-number"> <label for="wtu-phone-number"> <span>Phone number</span> </label> </div>
  <div class="form-group message-input"> <textarea class="form-control" id="wtu-message" name="Message" required="" rows="3"></textarea> <label for="wtu-message"> <span>Message <abbr class="required" title="required">*</abbr></span>
      <div class="invalid-feedback">Missing</div>
    </label> </div> <button class="btn btn-primary" type="submit">Send Message</button>
  <div class="overall-invalid-feedback" role="alert">Unable to submit. Please see errors above.</div>
  <p class="registration-terms">We collect and use this information in accordance with our <a href="/gdpr-privacy" target="_blank">privacy policy</a></p>
</form>

Text Content

We use cookies and similar technologies to ensure you get the best experience on
our websites. By actively using the website you're consenting to use these
cookies. Change cookie preferences
OK
Skip to main content

 * Documentation
 * Blog
 * Distributed systems design fundamentals training

Home page
 * Platform
   * Platform overview Everything in Particular
   * NServiceBus Messaging & workflow
   * ServiceInsight Advanced debugging
   * ServicePulse Production monitoring
     
   * Downloads
 * Solutions
   * Finance
   * Healthcare
   * Retail
 * Pricing
   * Pricing overview
   * Licensing definitions
   * Frequently asked questions
 * Support
   * Support options
   * Consulting
   * On-site training
 * Resources
   * Documentation
   * Learning path
   * Training courses
   * Webinars
   * Videos & presentations
     
   * Ask a question
   * Source code
   * Proof-of-concept help
   * Third-party licenses
 * Community
   * Events
   * Blog
   * NServiceBus Champs
   * Discussion group
 * Company
   * About us
   * Customers
   * Case studies & testimonials
   * Partners
   * Careers
   * Contact us
 * Get started


BANISH GHOST MESSAGES AND ZOMBIE RECORDS FROM YOUR WEB TIER

Written by Laila Bougria, Tomek Masternak, Daniel Marbach, Tim Bussmann, and
Szymon Pobiega on October 25, 2022

Because it’s hard to write idempotent code effectively, NServiceBus provides the
outbox feature to make your business data transaction and any sent or received
messages atomic. That way, you don’t get any ghost messages or zombie records
polluting your system.


1



But the outbox can only be used inside a message handler. What about web
applications and APIs?

With the new NServiceBus.TransactionalSession package, you can use the outbox
pattern outside of a message handler too.


🔗THE PROBLEM

Let’s say you have a web application where you need to create an entity and
perform some background processing.

Before the transactional session, the best guidance was to not do any database
work inside the ApiController, but to only take the input data and send a
message to the back end, responding only with an HTTP 202 Accepted message.
Then, a few milliseconds later, a message handler would pick up the message and
process it with the complete protection of the outbox feature.

But this isn’t always very realistic. What if the database is in charge of ID
generation, and you must return that ID to the client? Or do you need to update
the UI to show the request, even if the processing isn’t complete yet?


🔗GHOST PROTOCOL

This example code compromises by inserting a single record using Entity
Framework and then sends a message to the backend. As a result of the
compromise, this code is still vulnerable to ghost messages if the database
transaction has to roll back after the message has been sent.

[ApiController]
public class SendMessageController : Controller
{
    readonly MyDataContext dataContext;
    readonly IMessageSession messageSession;

    public SendMessageController(IMessageSession messageSession, MyDataContext dataContext)
    {
        this.messageSession = messageSession;
        this.dataContext = dataContext;
    }

    [HttpPost]
    public async Task<string> Post(Guid id)
    {
        await dataContext.MyEntities.AddAsync(new MyEntity { Id = id, Processed = false });

        var message = new MyMessage { EntityId = id };
        await messageSession.SendLocal(message);

        return $"Message with entity ID '{id}' sent to endpoint";
    }
}


Those familiar with Entity Framework might wonder where the call to
SaveChangesAsync is happening. Because Entity Framework already supports the
Unit of Work pattern, the calls to SaveChangesAsync can be done using ASP.NET
Core middleware, which means the database is only updated when the HTTP request
successfully completes:

public class UnitOfWorkMiddleware
{
    readonly RequestDelegate next;

    public UnitOfWorkMiddleware(RequestDelegate next, MyDataContext dataContext)
    {
        this.next = next;
    }

    public async Task InvokeAsync(HttpContext httpContext, ITransactionalSession session)
    {

        await next(httpContext);

        await dataContext.SaveChangesAsync();
    }
}


Regardless of the outcome of the database operations, the moment our controller
logic calls SendLocal, the message is handed over to the transport. This makes
the message MyMessage almost immediately available to be processed in the
background.

Things will often seem fine until a transient exception forces your Entity
Framework operations to roll back. The database entity was never committed, but
the ghost message starts processing in the background anyway. The message
handler tries to load the missing entity from the database…and disaster unfolds.

Imagine looking into this error and trying to figure out why that entity doesn’t
exist. Obviously, the message was sent, so the entity should be in the database,
but it’s not. Where is it?

Unfortunately, ghost messages don’t know they’re ghosts,


2 which makes them hard to diagnose.




🔗GHOSTBUSTERS

So how do we banish the ghost message? A proton pack won’t help you, but our new
TransactionalSession packages can.

In this case, since we’re using Entity Framework, we’ll use
NServiceBus.Persistence.Sql.TransactionalSession, but we’ve got a bunch of
different packages available depending on what database you’re using.

In our code, we replace IMessageSession with ITransactionalSession like this:

[ApiController]
public class SendMessageController : Controller
{
    readonly MyDataContext dataContext;
-   readonly IMessageSession messageSession;
+   readonly ITransactionalSession messageSession;

-   public SendMessageController(IMessageSession messageSession, MyDataContext dataContext)
+   public SendMessageController(ITransactionalSession messageSession, MyDataContext dataContext)
    {
        this.messageSession = messageSession;
        this.dataContext = dataContext;
    }

    // Rest omitted. Believe us, the code looks the same ;)
}


All the business logic stays the same. With that small change, the newly stored
entity and the sends are wrapped in an all-or-nothing transaction, meaning both
succeed or fail. The TransactionalSession package works with all the persistence
technologies supported by NServiceBus, like Microsoft SQL Server, CosmosDB, and
many more.

Check out this video demo of the TransactionalSession in action:


🔗ONE FINAL DETAIL

Much like the code that ensures Entity Framework operations are executed against
the database by calling SaveChangesAsync, there needs to be a tiny bit of
middleware that ensures the transactional session is committed when the HTTP
pipeline has to be executed.

public class UnitOfWorkMiddleware
{
    readonly RequestDelegate next;
+   readonly ITransactionalSession messageSession;

-   public UnitOfWorkMiddleware(RequestDelegate next, MyDataContext dataContext)
+   public UnitOfWorkMiddleware(RequestDelegate next, ITransactionalSession messageSession)
    {
        this.next = next;
+       this.messageSession = messageSession;
    }

    public async Task InvokeAsync(HttpContext httpContext, ITransactionalSession session)
    {

+       await session.Open(new SqlPersistenceOpenSessionOptions());

        await next(httpContext);

-       await dataContext.SaveChangesAsync();
+       await session.Commit();
    }
}


Notice how the middleware no longer calls SaveChangesAsync anymore–instead, the
middleware commits the transaction on the data context. The transaction created
by the transactional session will take care of saving all database changes and
triggering the outbox so that everything remains consistent and atomic.


🔗BULLETPROOF

The algorithm behind the Transactional Session feature was based on the proven
NServiceBus Outbox implementation. But we’ve also modeled and verified the
algorithm using TLA+,


3 a formal specification language to verify and test programs. Plus, we’ve
covered it with a rich set of automated test suites covering every supported
database engine, so you know you can trust it.




🔗SUMMARY

With the new TransactionalSession, there are no more compromises. You don’t have
to painfully redesign a web application to move all the data transactions to the
backend. Instead, you can update the database and send a message, and be
confident that the outbox implementation will prevent ghost messages or zombie
records.

To get started, check out the TransactionalSession documentation, including a
detailed description of how it works. Or, check out our Using
TransactionalSession with Entity Framework and ASP.NET Core sample to see how to
use it in your own projects.

Share on Twitter


ABOUT THE AUTHORS

Laila is a software engineer who's terrified of ghost messages.

 * Twitter
 * LinkedIn

Tomek is a software engineer who prefers things to occur exactly once.

 * Twitter
 * LinkedIn

Daniel is a software engineer who strives to be eventually consistent.

 * Twitter
 * Personal blog
 * LinkedIn

Tim is a software engineer who highly values idempotency. Tim is a software
engineer who highly values idempotency.

Szymon is a software engineer who loathes zombie records.

 * Twitter
 * Personal blog
 * LinkedIn


RELATED CONTENT

 * RPC vs. Messaging – which is faster?
 * Cross-platform integration with NServiceBus native message processing
 * Autosave for your business
 * What does idempotent mean?
 * What Starbucks can teach us about software scalability

--------------------------------------------------------------------------------

 1. For details on ghost messages, zombie records, and why they pose a problem
    for distributed systems, check out our blog post What does idempotent mean?

 2. Which is sad for the messages, I guess?

 3. …which you can learn about in our webinar Implementing an Outbox –
    model-checking first.

Don't miss a thing. Sign up today and we'll send you an email when new posts
come out.
Thank you for subscribing. We'll be in touch soon.
  Get notified
We collect and use this information in accordance with our privacy policy.

Blog home
RSS feed
Webinar: An exception occurred... Please try again. with Laila Bougria
If there is one certainty in software, it’s that things fail. See how to embrace
those failures.
Learn more →
NServiceBus Quick Start
Learn why software systems built on asynchronous messaging using NServiceBus are
superior to traditional synchronous HTTP-based web services.
Check it out →
Contact us

TELEPHONE NUMBERS:

US toll-free:(888) 495-6156 New York:(646) 741-8577 London:(020) 33189097
Sydney:(02) 80466544 Tel Aviv:(03) 721-9577 Write to us
 * Platform
 * Platform overview
 * NServiceBus
 * ServiceInsight
 * ServicePulse
 * Downloads

 * Solutions
 * Finance
 * Healthcare
 * Retail

 * Pricing
 * Pricing overview
 * Licensing definitions
 * Frequently asked questions

 * Support
 * Support options
 * Consulting
 * On-site training

 * Resources
 * Documentation
 * Learning path
 * Training courses
 * Webinars
 * Videos & presentations
 * Ask a question
 * Source code
 * Proof-of-concept help
 * Third-party licenses

 * Community
 * Events
 * Blog
 * NServiceBus Champs
 * Discussion group

 * Company
 * About us
 * Customers
 * Case studies & testimonials
 * Partners
 * Careers
 * Contact us

© 2010-2022 NServiceBus Ltd. doing business as Particular Software. All rights
reserved.
 * Privacy policy
 * GDPR privacy policy
 * Cookie Settings
 * Terms of Use

Need help getting started?

WRITE TO US

×

Required fields are marked with *

First name *
Missing
Last name *
Missing
Company name
Email address *
Invalid
Phone number
Message *
Missing
Send Message
Unable to submit. Please see errors above.

We collect and use this information in accordance with our privacy policy

Thank you for your message. We will get back to you soon.