The old NatNeg thread - How to avoid bouncing in Pitboss (without Gamespy)

[novice]

Hello,

probably I know an ultimate solution which do not need any entries in the hosts file. We only need one fixed domain which we to connect Caledorn's server (or another with almost 100% uptime). Then, we patch the exe and replace the gamespy url's (just a few bits) with the new domain name. Everyone, who use this new Exe should never need any changes.
The usage of a new exe should be a tiny drawback. (It's no need to overwrite the originial file…)

Edit: I've done a test with a domain of me and it works as desired.

Regards Ramkhamhaeng (from civforum.de)

P.S. We working currenty on a Mod which fixes some other bugs off pitboss servers, but it's to early to present it here.

That's an interesting idea, although I don't know what the acceptance level for a hacked exe file would be. I would personally rather just edit my hosts file now and then. I suppose both approaches could be used in parallel though.

Regarding your P.S, slightly off topic:

We've also fixed a few pitboss issues in the CvGameCoreDLL.dll:
* Autosaves are created at login, end turn, and log out.
* Diplo window doesn't show up when game is paused (shows up when game is unpaused again)
Also a few general bug fixes:
* Trade route turn order bug
* Foreign trade route cities lost permanently when your city using those
routes is destroyed
* Build culture double production
* Build wealth/research/culture + production automation double
production
* Proposed trades including cities, where the cities no longer exist to
be trades, are not cancelled
* Production decay counter on a type of build (e.g. axeman) is not reset
after completing one if the next item in the queue is of the same type
* Feature growth/disappearance rates, and bonus discovery (mine pop)
rates, do not scale with game speed

The source code for that dll is here:
https://github.com/novice-rb/Civ4RealmsBeyondBalanceMod

The dll contains other functionality too though related to event logging, an observer mode, and various xml-triggered balance changes.

[Zulan]

Here are a few more thoughts:

Other dead hosts:

To avoid long timouts (especially on linux/wine) in the Pitboss creation, I also reassigned "motd.gamespy.com" to an ip where a HTTP server runs. It will request this "/motd/vercheck.asp?productid=11081&versionuniqueid=3.1.9.0+(128100)&distid=0&gamename=civ4bts", it doesn't seem to matter if a 404 comes.

Stable natneg hosting:

I would be willing to run a server with a stable ip. I just updated the hardware - previously I had an uptime of >1500 days before I had to retire it due to old age. Small catch - it is not a failover IP, so I cannot move it to a new server if the hardware at some point is too old again. It is running Linux (Ubuntu 14.4) - is there any experience running the natneg in Linux/wine/mono?

Ideally we (as a community) provide multiple servers for (especially long term) redundancy. This way if one server is down (temporary) players don't have to change their hosts/exe. However, I wonder how good the redundancy with the 3 dns is implemented in Civ4? Will it quickly try the 3 hosts or run into long timeouts first?

Possible solutions:

• Have Gamespy.com setup DNS A records for us. While not very likely, ihis is clearly the best solution. I think this should not be discarded. Maybe we can find a sympathetic techie at gamespy?
  
• Have 2k patch the game for us (configurable settings in ini). I think rather unlikely due to the effort and actually also not that nice because the autoupdate doesn't work.
  
• Modify the exe. Not elegant, but I suppose this is easy and pragmatic. That way we can change the IP - the domains are much more long-term available than IPs. I also have a short-enough domain name if needed.
  
• Modify /etc/hosts. Maybe with help of a small tool. Unfortunately there seem to be quite some problems with caching and/or anti-virus.
  

[Zulan]

P.S. Does anyone know if there is a difference in functionality between the civ4bts.natneg*.gamespy.com and civ4bts.available.gamespy.com?

[novice]

Other dead hosts:

To avoid long timouts (especially on linux/wine) in the Pitboss creation, I also reassigned "motd.gamespy.com" to an ip where a HTTP server runs. It will request this "/motd/vercheck.asp?productid=11081&versionuniqueid=3.1.9.0+(128100)&distid=0&gamename=civ4bts", it doesn't seem to matter if a 404 comes.

Yes, we're aware of this and redirecting to 127.0.0.1 works well - this was admittedly on a computer with a local web server. At any rate, this is only needed on the pitboss server, not on the client computers.

Is there any experience running the natneg in Linux/wine/mono?

No, but it would be great if you tested that - just get the code or the binary from this thread.

Ideally we (as a community) provide multiple servers for (especially long term) redundancy. This way if one server is down (temporary) players don't have to change their hosts/exe. However, I wonder how good the redundancy with the 3 dns is implemented in Civ4? Will it quickly try the 3 hosts or run into long timeouts first?

It pings all three hosts immediately (by sending them an INIT message). Servers 2 and 3 do nothing except answer the ping though, I don't know what the exact failover mechanism is and as such it isn't implemented. Server 1 is the only one that does actual NAT Negotiation currently. It is possible that all servers could act as if they were server 1 and that this would work, it is also possible that this would just confuse the clients. I haven't tested.

P.S. Does anyone know if there is a difference in functionality between the civ4bts.natneg*.gamespy.com and civ4bts.available.gamespy.com?

There's a difference. The civ4bts.available communication uses port 27900 whereas the natneg communication uses port 27901. 27900 is the gamespy master server, I just used Project Reality's implementation here. Civ doesn't seem to do anything except ping that server.

[Ramkhamhaeng]

That's an interesting idea, although I don't know what the acceptance level for a hacked exe file would be. I would personally rather just edit my hosts file now and then. I suppose both approaches could be used in parallel though.

Thats true. To change the exe would be a practical solution if you want play on systems where you don't have administration rights. (Until now, it was nice/easy to use Civ as protable Game on a usb stick.)

Regarding your P.S, slightly off topic:

Some of the non-gameplay fixes looks eligible
I discuss to adapt this patches in your mod after we solve the Gamspy problem.

[Zulan]

No, but it would be great if you tested that - just get the code or the binary from this thread.

I built the code on Windows VS2012 Expess, ran into one issue:

Code:

Unhandled Exception:
System.NotImplementedException: The requested feature is not implemented.
  at System.IO.MemoryMappedFiles.MemoryMapImpl.Open (System.String path, FileMode mode, Int64 capacity, MemoryMappedFileAccess access) [0x00000] in <filename unknown>:0
  at System.IO.MemoryMappedFiles.MemoryMappedFile.CreateFromFile (System.String path, FileMode mode, System.String mapName, Int64 capacity, MemoryMappedFileAccess access) [0x00000] in <fi  lename unknown>:0
  at MaxMind.Db.Reader..ctor (System.String file, FileAccessMode mode) [0x00000] in <filename unknown>:0
  at MaxMind.GeoIP2.DatabaseReader..ctor (System.String file, System.Collections.Generic.List`1 locales, FileAccessMode mode) [0x00000] in <filename unknown>:0
  at MaxMind.GeoIP2.DatabaseReader..ctor (System.String file, FileAccessMode mode) [0x00000] in <filename unknown>:0
  at PRMasterServer.Data.GeoIP.Initialize (System.Action`2 log, System.String category) [0x00000] in <filename unknown>:0
  at PRMasterServer.Servers.ServerListReport..ctor (System.Net.IPAddress listen, UInt16 port, System.Action`2 log, System.Action`2 logError, System.String gameName) [0x00000] in <filename   unknown>:0
  at PRMasterServer.Program.Main (System.String[] args) [0x00000] in <filename unknown>:0
[ERROR] FATAL UNHANDLED EXCEPTION: System.NotImplementedException: The requested feature is not implemented.
  at System.IO.MemoryMappedFiles.MemoryMapImpl.Open (System.String path, FileMode mode, Int64 capacity, MemoryMappedFileAccess access) [0x00000] in <filename unknown>:0
  at System.IO.MemoryMappedFiles.MemoryMappedFile.CreateFromFile (System.String path, FileMode mode, System.String mapName, Int64 capacity, MemoryMappedFileAccess access) [0x00000] in <fi  lename unknown>:0
  at MaxMind.Db.Reader..ctor (System.String file, FileAccessMode mode) [0x00000] in <filename unknown>:0
  at MaxMind.GeoIP2.DatabaseReader..ctor (System.String file, System.Collections.Generic.List`1 locales, FileAccessMode mode) [0x00000] in <filename unknown>:0
  at MaxMind.GeoIP2.DatabaseReader..ctor (System.String file, FileAccessMode mode) [0x00000] in <filename unknown>:0
  at PRMasterServer.Data.GeoIP.Initialize (System.Action`2 log, System.String category) [0x00000] in <filename unknown>:0
  at PRMasterServer.Servers.ServerListReport..ctor (System.Net.IPAddress listen, UInt16 port, System.Action`2 log, System.Action`2 logError, System.String gameName) [0x00000] in <filename   unknown>:0
  at PRMasterServer.Program.Main (System.String[] args) [0x00000] in <filename unknown>:0

Fortunately that was easy to fix. In Data/GeoIP.cs ::Initialize:

Code:

            if (File.Exists("GeoIP2-Country.mmdb")) {
                reader = new DatabaseReader("GeoIP2-Country.mmdb", MaxMind.Db.FileAccessMode.Memory);
                log(category, "Loaded GeoIP2-Country.mmdb");
            } else if (File.Exists("GeoLite2-Country.mmdb")) {
                reader = new DatabaseReader("GeoLite2-Country.mmdb", MaxMind.Db.FileAccessMode.Memory);
                log(category, "Loaded GeoLite2-Country.mmdb");
            } else {

Doesn't seem to a be a problem, uses 43 MB memory. I suppose you could also add a try/catch if you wanted to keep the MemoryMapped default behavior.

There are no suspicious errors, and the ports are open:

Code:

tcp        0      0 *:29900                 *:*                     LISTEN      9967/PRMasterServer
tcp        0      0 *:29901                 *:*                     LISTEN      9967/PRMasterServer
tcp        0      0 *:28910                 *:*                     LISTEN      9967/PRMasterServer
udp        0      0 *:29910                 *:*                                 9967/PRMasterServer
udp        0      0 *:27900                 *:*                                 9967/PRMasterServer

So I am now running the natneg at

148.251.130.188

a.k.a.

civ4bts.natneg1.g.zulan.net
civ4bts.natneg2.g.zulan.net
civ4bts.natneg3.g.zulan.net
civ4bts.available.g.zulan.net
motd.g.zulan.net

I would appreciate if someone could test this to see if it is actually working . This is a stable IP and reliable server, I intend to keep this running for approx. 4-5 Years - assuming the program runs stable and securely. I might possibly move it to a different IP on that Server if there are some technical issues.

[Ramkhamhaeng]

So I am now running the natneg at

148.251.130.188

a.k.a.

civ4bts.natneg1.g.zulan.net
civ4bts.natneg2.g.zulan.net
civ4bts.natneg3.g.zulan.net
civ4bts.available.g.zulan.net
motd.g.zulan.net

I would appreciate if someone could test this to see if it is actually working . This is a stable IP and reliable server, I intend to keep this running for approx. 4-5 Years - assuming the program runs stable and securely. I might possibly move it to a different IP on that Server if there are some technical issues.

And here are some patched executables for this dns entries: Civ4BeyondSword2014.exe Civ4BeyondSword_PitBoss2014.exe


Regards

[novice]

Code:

PRMasterServer.exe +game civ4bts +servers master,natneg

The server listens on ports 27900 and 27901 for UDP traffic, so you need to let UDP traffic on those ports through any firewalls and routers you might have.

You should only need to run the master server, which listens on port 27900, and the natneg server, which listens on port 27901. Use the startup parameters described above.

[Zulan]

Code:

PRMasterServer.exe +game civ4bts +servers master,natneg

The server listens on ports 27900 and 27901 for UDP traffic, so you need to let UDP traffic on those ports through any firewalls and routers you might have.

You should only need to run the master server, which listens on port 27900, and the natneg server, which listens on port 27901. Use the startup parameters described above.

Done, works as described

Code:

udp        0      0 *:27900                 *:*                                 15650/PRMasterServe
udp        0      0 *:27901                 *:*                                 15650/PRMasterServe

[novice]

Thanks for your effort, Zulan.

Has anybody been able to test Zulan's natneg server? Use this ip address in your hosts file to test: 148.251.130.188.

If the server works it's probably preferrable to Caledorn's storm-ridden server, although there's no point unless everybody starts using it. At the very least it would be good to verify that it works and having it available as a backup natneg server.