Build without VS2017

[igogrek]

Hi.
Great that you were able to refactor all of the different DLL's into one solution.
It's much cleaner and managable now.

The question: is it possible to build without VS2017? I'm talking about stuff like VScode for example.

Because currently I've tried running build.bat and had numeroues issues, like:

C:\Projects\Booma.Library\src\Booma.GameServerList.Service\Booma.GameServerList.Service.csproj : error NU1605: Detected package downgrade: Microsoft.EntityFrameworkCore.Design from 1.1.2 to 1.1.1. Reference the package directly from the project to select a different version. \r [C:\Projects\Booma.Library\Booma.Library.sln]
C:\Projects\Booma.Library\src\Booma.GameServerList.Service\Booma.GameServerList.Service.csproj : error NU1605: Booma.GameServerList.Service (>= 1.0.0) -> Microsoft.EntityFrameworkCore.Tools (>= 1.1.1) -> Microsoft.EntityFrameworkCore.Design (>= 1.1.2) \r [C:\Projects\Booma.Library\Booma.Library.sln]
C:\Projects\Booma.Library\src\Booma.GameServerList.Service\Booma.GameServerList.Service.csproj : error NU1605: Booma.GameServerList.Service (>= 1.0.0) -> Microsoft.EntityFrameworkCore.Design (>= 1.1.1) [C:\Projects\Booma.Library\Booma.Library.sln]

And than

Booma.Payloads.Instance -> C:\Projects\Booma.Library\src\Booma.Payloads.Instance\bin\release\net46\Booma.Payloads.Instance.dll
IoC\NetworkMessageHandlerServiceRegistration.cs(1,7): error CS0246: The type or namespace name 'GladNet' could not be found (are you missing a using directive or an assembly reference?) [C:\Projects\Booma.Library\src\Booma.Unity.Common\Booma.Unity.Common.csproj]
IoC\NetworkMessageHandlerServiceRegistration.cs(2,7): error CS0246: The type or namespace name 'GladNet' could not be found (are you missing a using directive or an assembly reference?) [C:\Projects\Booma.Library\src\Booma.Unity.Common\Booma.Unity.Common.csproj]
IoC\NetworkMessageHandlerServiceRegistration.cs(3,7): error CS0246: The type or namespace name 'GladNet' could not be found (are you missing a using directive or an assembly reference?) [C:\Projects\Booma.Library\src\Booma.Unity.Common\Booma.Unity.Common.csproj]
IoC\NetworkMessageHandlerServiceRegistration.cs(4,7): error CS0246: The type or namespace name 'SceneJect' could not be found (are you missing a using directive or an assembly reference?) [C:\Projects\Booma.Library\src\Booma.Unity.Common\Booma.Unity.Common.csproj]

I assume that the problem is most likely related to Nuget or something (and it works differently from VS2017). Or maybe build.bat is out of date or something?

[HelloKitty]

GladNet references are from this nuget feed https://www.myget.org/F/hellokitty/api/v3/index.json so you may not be able to get GladNet or the latest sceneject without adding that feed to VS. I think it's in the NuGet config which I thought VS looked at when populating feeds.

I haven't tried it with anything below VS2017. I'll try a clean clone later today to see what could be wrong but other than that I can only say I use VS2017 and I'm not sure of the differences.

[HelloKitty]

Oh it completely escaped me! You will not be able to build the full solution because a dll in the lib folder is missing. We started to use Odin Inspector but it's not opensource and requires a license.

We can't redistribute all the DLLs required to build the solution anymore. It wasn't a decision made lightly but Odin replaced an old library I wanted to stop maintaining, GladBehaviour, and it improved the ability to serialize to the Unity inspector and allowed me to work with generics abit more.

So that could be another reason for the failures!

[igogrek]

Yeah, would be very nice to build the project only using CLI. This way it could be developed even on non-windows platforms, without VS2017.

There are at least 2 possible ways I see to solve Odin Inspector DLL issue:

1. Make it only needed for "designer" build and remove usages from core projects.
2. Make a dummy DLL without real implementations, I've used such approach before (maybe this dummy exists already?)

[HelloKitty]

I have considered a dummy dll that can be committed. That's a reasonable solution. The resulting compiled assemblies won't be usable in Unity3D though if that is done.

Also, as of recently there has been a major pivot in the project. To reduce scope, increase chance of adoption and tap into the vast domain knowledge the PSO community developers and reverse engineers have I've decided to start laying the groundwork for a custom PSOBB client.

I've already started working with someone very knowledgeable in this area and we've been putting together something rather quickly.

It's more greenfield and collaboration has and will be a key goal. Starting from day 1 this time. https://github.com/HelloKitty/Booma.Proxy

[igogrek]

Nice!
Reworking whole existing PSOBB server sounds much more feasible.
This way more work can be spent on actually writing a nice new client.
Would be glad to help with either one.

[HelloKitty]

@igogrek Ah the beauty of this reduction in scope is that we will not be building a PSOBB server. Compatibility with Tethella, and by extension Ultima and Ephenia, is the main goal. We're only going to focus on building a client.

However you could build a server with the libraries being implemented right now. There just isn't a use for another PSOBB server implementation I think. A new client unlocks more possibilities.

[HelloKitty]

You're welcome to contribute. I accept PRs. Requires a lot of reading of Sylverant or Tethella or existing domain specific knowledge. Check out the packet csprojs to see how they're created. The main effort is going towards creating DTOs for all the packets.

Lots of wide open space for that. Refactoring can always be done once when the main structure has been established into things like nested objects, enums or etc. So contributing the structures for any packets can be helpful.

It's late so I have to sleep though, hope we end up collabing!

[igogrek]

I wish I could just jump in but in the absence of docs it's going to be quite problematic.
Shame that SodaBoy can't/don't want to share the sources he has for server now.

Found the run instructions for the server setup and will try to do so later.
I agree that DTO's should be first thing to start. I thought that the fastest way to proceed would be sniffing the packets between client and server and get the full understanding on what is going on where.
But there will be an encryption...

At least Sylverant's code seem to be commented, but, again, it's hard to go just from the sources, without seeing the full picture.

[HelloKitty]

The best documentation I can provide is: https://sylverant.net/wiki/index.php/Category:PSO_Server_Protocol

It's incomplete and contains DC, PC, GC and BB information.

Aside from that I use Sylervant's PSOBB login/patch server implementation: https://github.com/Sylverant

Tethella doesn't have a recent public release, and I don't blame Sodaboy for that decision, but there is an old Tethella source here https://github.com/justnoxx/psobb-tethealla but it's less helpful than Sylverant I think. Also Ephenia's version of Tethella uses a different encryption than the standard blowfish from the original PSOBB.

Going from these source and Soly's understanding of the client is all we have to go on. I rely on Soly for dumping packets since I don't have a working proxy nor do I run my own server to dump the incoming packets.

We have a working implementation for the patchserver encryption, check out the repo, and we have a probably working blowfish implementation for the login/ship server encryption. It's a lot, but we're slowing working through it all.

[HelloKitty]

@igogrek I've tried my hand at generating some documentation for packets with a tool I wrote. Looking for feedback on it if there is more meaningful information you thing should or could be produced automatically to help potential contributors or people looking to understand the project.

https://github.com/HelloKitty/Booma.Proxy/tree/master/docs

[igogrek]

Oh, that's https://sylverant.net/wiki/index.php/Category:PSO_Server_Protocol exactly what I was looking for!

https://github.com/HelloKitty/Booma.Proxy/tree/master/docs looks like it should.
Next thing would be to add all required opcodes for other servers here (from wiki above) with TODO's.

Few questions here:

1. Do we need PC packets? Are they required even if we only want to emulate PSOBB? What about DC?
2. Hard to read the wiki always jumping from shorthand hex to the full, let's stick with longer one.
3. Am I right and we can't actually rely on wiki only? I only see 0x0014 and 0x0614 in the sources...
4. Obviously from sources it seems that Ship Server will be the last thing to do, but what's the actual order? Patch, Login, Ship? Sylverant also has central so-called 'shipgate'.

Some answers to these questions may be useful for future contributors, so wiki page with this sort of information would be nice. FAQ/overview or something. With all of the great links and general idea you've wrote here already.

But to actually work with this, I think we need some kind of debug/test tool.
Maybe even real PSOBB client to check if it is able to connect or something.
As of now, I got much more sources understanding, but still wondering from which side I can tackle all of this (I think that would be the case for any future contributor).

[HelloKitty]

Sylverant is incomplete for PSOBB. It seemed like the goal had been crossplatform play, similar to the old PSO PC days where you could play with DC players,

I'm working on generating enumerations for the login and ship opcodes. Once we have those then the doc gen tool can produce similar files too. I don't have those, but I'm waiting for a login server dump from my other collaborator to start putting one together.

1. I don't think we need PC packets, though they may share some formats or structures. I'm not yet sure.
2. Like I said Sylverant is incomplete and its documentation is less complete. Use of 4 length hex string is due to the fact that opcodes are 2 bytes. Not sure if any enter the range of 2 bytes though other than 0x0614 so far.
3. We really can't rely on anything as Scht, Ephenia and everyone does something different. No opensource PSOBB project is/was complete either.
4. The flow of client appears to be connecting to Patch server, which has its own auth, then the patch server redirects to a patch file server. After finishing the client manually connects to the login server and then the ship servers after that. Shipgate may refer to the service that sits inbetween login and ships since ships may be regional and not on the same IP. It may be something like application level DNS for routing to ships users select on the ship screen. But I don't know, I don't have too much domain knowledge about PSO. I merely implement to specification.

I currently don't do debugging nor do we yet run tests using realworld dumps. I want to, but it takes up development time. In a perfect world we would have test cases covering that all that all real world data could be serialized and then deserialized to and from the DTOs. Its just implementing these things have a cost. One we're not 100% sure yet has value since there is so much work to be done already.

Right now I rely on Soly to dump interactions between the client and server. I have written up a simple test client you can find in the Tests subdir that will connect to non-Ephenia servers. It's not ideal but it's the biggest bank for the least buck.