A transaction is a SIP message exchange between two user-agents that starts with a request and ends with its final response (it can also contain zero or more provisional responses in between). For example, during the termination of a SIP session, one user releases the call by sending a BYE request and the other party replies back with a 200 OK response. This message exchange is called a transaction. Transactions are actually part of a dialog.
A dialog is a complete exchange of SIP messages between two user-agents. That means that transactions are actually part of a dialogA dialog is a peer-to-peer SIP relationship between two UAs that persists for some time. A dialog is established by SIP messages, such as a 2xx response to an INVITE request. A dialog is identified by a call identifier, local tag, and a remote tag.