Difference between revisions of "Packets"
(added link to Opcodes documentation page) |
|||
| Line 113: | Line 113: | ||
The "opcode" in this case is "pay the bill, Tom, or we'll shut your power off!" | The "opcode" in this case is "pay the bill, Tom, or we'll shut your power off!" | ||
| − | Tom now must sit down at his desk and write a check for the amount due, put it in the envelope and return it to the Power Company | + | Tom now must sit down at his desk and write a check for the amount due, put it in the envelope, and return it to the Power Company |
| − | before due date. | + | before the due date. |
In this example, you can see how simple it really is when you strip away all the networking, encryption, hexadecimal characters, and other frightening aspects of data packets. | In this example, you can see how simple it really is when you strip away all the networking, encryption, hexadecimal characters, and other frightening aspects of data packets. | ||
For more details, see [[Developer:Opcodes|Opcodes]]. | For more details, see [[Developer:Opcodes|Opcodes]]. | ||
| + | |||
| + | Someone started an [[Opcodes|Opcodes]] documentation but doesn't look like any work was actually done on it. | ||
Latest revision as of 19:01, 27 February 2022
Developers - Packet Resources
Return to: Developers | Tutorials | Portal | Forum | Project Manager | Bug Tracker
I always try to find simple, real-world analogies to help me better understand complex computer concepts. The one I use for client->server is the US Postal Service (we do love them, do we not?)
You (client) write a letter (data packet), stick it in an envelope and address it to your friend Tom Jones including his street address. The Postal Service (network communication layer) delivers it to the address on the envelope (server). The residents of that address fetch the letter and see it is addressed to Tom (opcode) and hand it to him so he can read it (data) and respond accordingly.
See? Simple 
In this category, we will attempt to document the well-known, critical packets we rely on in the hopes others will begin to understand the mystery and join in our efforts to develop the server to completion.
This Wiki was written based on the following forum post: Updating Opcodes by LethalEncounter.
The major classifications for packet types in EQ2Emulator are:
Server Packets : Data that leaves the server destined for the client Client Packets : Data that leaves the client destined for the server
Consider Client/Server packets the root of the tree as we start understanding packets and how they work.
From there, we can further break out packet types (specific to EQ2Emulator) in more detail as follows:
static const char OP_SessionRequest = 0x01; static const char OP_SessionResponse = 0x02; static const char OP_Combined = 0x03; static const char OP_SessionDisconnect = 0x05; static const char OP_KeepAlive = 0x06; static const char OP_ServerKeyRequest = 0x07; static const char OP_SessionStatResponse= 0x08; static const char OP_Packet = 0x09; static const char OP_Fragment = 0x0d; static const char OP_OutOfOrderAck = 0x11; static const char OP_Ack = 0x15; static const char OP_AppCombined = 0x19; static const char OP_OutOfSession = 0x1d;
The only type we are interested in is OP_Packet (09). This is the type that you need to look for inside of the packet log.
Note: However, Packet Collector will sometimes strip the 00 09 off depending on how the packet was sent. More on this in The Packet Log.
It is important to remember that all packet data is in hexadecimal format, and is rarely in "plain English" so you will need a calculator or Hex converter utility to see the actual values in the packet.
Critical!
Also, most of the values are in Little Endian format. This can be most confusing to those not familiar with network data or programming in general, but once you grasp the concepts it will all start making sense. Read this article on Endianness if you are curious what this is all about.
Lastly, some of the data packets coming from the server are "compressed" and must be "unpacked" before they can be observed properly. More on this later.
All data packets have a header that contain important information on what is inside the packet; things like packet type (above), sequence, whether it is compressed or not, and the OPeration CODE.
Example:
-- OP_ClientCmdMsg::OP_EqCreateGhostCmd -- 0000: 00 09 00 1E 00 37 7F 00 00 00 FF E5 01 01 03 00 .....7.......... 0010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 0020: 00 00 00 00 00 00 00 00 48 00 00 00 8F FF FF FF ........H....... 0030: FF 10 E5 40 80 D3 D3 C4 02 04 1D 81 11 0B 95 20 ...@........... 0040: 20 20 0B 97 14 04 01 03 82 06 81 70 7F 7F 1D CF .........p.... 0050: 04 06 01 01 11 81 04 08 F5 1B 38 63 3D 3D 7F 7F ..........8c==.. 0060: 0B 87 37 9E 09 55 8D 04 28 5E 86 0D 80 12 81 0A ..7..U..(^...... 0070: 09 83 01 0D 07 00 44 69 61 74 65 73 74 00 00 00 ......Diatest... 0080 00 00 00 00 00 00 00 00 00 .........
The header, broken out:
00 09 : packet type 00 1E : sequence 00 : not compressed 37 : OP_ClientCmdMsg for this version 7F 00 00 00: packet size FF : Opcode is next two bytes instead of one E5 01 : Opcode (OP_EqCreateGhostCmd)
Everything after that is the packet data.
More detailed Packet Header breakdown:
1. The 00 09 is the Packet Type we're interested in.
2. The next 2 bytes in the packet is usually the sequence (00 1E).
The server and client use this to determine if a packet was lost and needs to be resent.
Only again, depending on how the packet was sent, Packet Collector might strip this off.
3. The next byte is a 1 if it is compressed using zlib or a 0, if not.
4. Now we get to the opcode of the packet. This is what determines what type of packet we are looking at. There are a few rules for opcodes:
A. If the opcode is less than 255, it will be a single byte, however if the opcode is greater than 254,
a 0xFF will be the first byte here, followed by the opcode which is now two bytes.
B. Once you have the opcode determined from the rules above it gets even more complicated.
If the opcode is a OP_ClientCmdMsg (depending on the version number this value changes),
then the next 4 bytes are the size of the packet, and finally the real opcode is listed just like in rule A.
Still with me?
Now we get into the meat of the packet -- the Data! Followed immediately after the opcode is the packet data. This is where it gets interesting. SOE uses various compression methods to make their packets as small as possible. For more info on Packed packets, see below Unpacking Packed Packets.
Opcodes are the link between the client and the server in respect to "what do you want me to do now?" A client's data packet is sent to the server with an Opcode in the header to tell the server exactly what the packet is, and what data it contains.
Taking our Postal Service example above, the data packet arrives at from the client to the server and includes an "opcode" which is a specific action to perform.
Simple example:
The letter Tom Jones received is his monthly power bill. The data within this "packet" is the amount and the date it is due. The "opcode" in this case is "pay the bill, Tom, or we'll shut your power off!" Tom now must sit down at his desk and write a check for the amount due, put it in the envelope, and return it to the Power Company before the due date.
In this example, you can see how simple it really is when you strip away all the networking, encryption, hexadecimal characters, and other frightening aspects of data packets.
For more details, see Opcodes.
Someone started an Opcodes documentation but doesn't look like any work was actually done on it.
Server Packets consist of more than just EQ2 Game Server data. For both SOE and EQ2Emulator, everything from Login to client patcher to any external access from within the client (Station Exchange, Legends of Norrath, etc) are all "servers" that respond to requests from a client.
Packets related to Server responses to Client requests are found here: Server Packets
Client packets are those generated on your computer while you are in the EQ2 Client playing the game. Whenever you click on something, open a UI screen, or interact with anything in-game - including just moving around - you are generating Client->Server packets.
Easiest example of this is when you move, your client builds an Appearance packet that is sent to the game server, which is then used to update your position not only for your own benefit (location triggers, for example) but also to show other players your character is moving -- this is a multi-player game, after all.
Packets related to Client responses to Server requests are found here: Client Packets
One method SOE uses to make their packets as small as possible we refer to as 'Packed'. The way this algorithm works is that it removes all the 0's from a packet thereby decreasing the size considerably since most packets are mostly 0s. They also reverse the order of the data in Packed packets (presumably to make it harder to decipher). During development, we need to unpack the packed packets in order to build the structs to support the data (reverse engineering the data).
There is a (relatively) simple way to unpack a packet and display it in the EQ2World console output. If we already know the data struct (XML) of a packet, we must correlate our data with the Struct, and if the data is not lining up, we will have to determine if it is Packed. If it is, then we must Unpack it before we can use it. There is an easy way to do this, using the function Unpack() in the EQ2Emulator C++ code:
Example of a PACKED packet:
-- OP_UpdateInventoryMsg -- 0000: 00 09 00 24 00 3F 01 00 2A 00 00 00 93 F2 38 22 ...$.?..*.....8" 0010: A0 07 B0 01 92 9F F2 FF 01 0A 01 FE 6D 6F 74 74 ............mott 0020: 6F 42 FF 61 42 20 73 73 65 6C FF 6E 45 20 66 6F oB.aB ssel.nE fo 0030 20 67 83 79 76 36 00 g.yv6.
The reason we know this is Packed is that the string text seen is backwards, and illegible. This is just one simple example, many are more complex (see The CharacterSheet packet)
For this specific example, look at the WS_UpdateInventory struct in WorldStructs.xml, and see that the first 2 bytes in this packet is item_count (01 00, little endian, remember?). Next is the packed_size (2A 00 00 00 - int32) which is where we always start when Unpacking a packet.
Code to Unpack it (add to top of main method in net.cpp):
uchar blah[] ={0x2A,0x00,0x00,0x00,0x93,0xF2,0x38,0x22,0xA0,0x07,0xB0,0x01,0x92,0x9F,0xF2,0xFF
,0x01,0x0A,0x01,0xFE,0x6D,0x6F,0x74,0x74,0x6F,0x42,0xFF,0x61,0x42,0x20,0x73,0x73
,0x65,0x6C,0xFF,0x6E,0x45,0x20,0x66,0x6F,0x20,0x67,0x83,0x79,0x76,0x36,0x00};
uchar* blah2 = new uchar[150];
Unpack(blah, blah2, 150, 1);
DumpPacket(blah2, 150);
delete[] blah2;
cout << "Done!\n";
First, a word about converting the packet data into 0x00 Hex codes required by the Unpack() function. You either have to manually change all the two-digit hex values to the 4-digit proper hex format (ie., 00 01 turns to 0x00,0x01) or you can use a tool written by Scatman called the EQ2EmuPacketTool.
The result of the output above:
0: 22 38 00 00 F2 00 00 00 - 00 00 00 00 07 00 00 00 | "8.............. 16: 00 00 92 01 00 01 0A 01 - FF F2 00 00 00 42 6F 74 | .............Bot 32: 74 6F 6D 6C 65 73 73 20 - 42 61 67 20 6F 66 20 45 | tomless Bag of E 48: 6E 76 79 00 00 00 00 00 - 00 00 00 00 00 00 00 00 | nvy............. 64: 00 00 00 00 00 00 00 00 - 00 00 00 00 00 00 00 00 | ................ 80: 00 00 00 00 00 00 00 00 - 00 00 00 00 00 00 00 00 | ................ 96: 00 00 00 00 00 00 00 00 - 00 00 00 00 00 00 CD CD | ................ 112: CD CD CD CD CD CD CD CD - CD CD CD CD CD CD CD CD | ................ 128: CD CD CD CD CD CD CD CD - CD CD CD CD CD CD CD CD | ................ 144: CD CD CD CD CD CD | ......
This is the data that you can actually use. The output size (eg., 150 above) changes for each packet, just use a big enough size so you start seeing the 0xCD bytes. The 0xCD's at the end are just uninitialized bytes which you can ignore. If you don't see any 0xCD bytes on the end then your blah2 variable is too small. Increase it and try again.