New Home for the CCG Toolkit

CCG Toolkit

With the transfer of the blog and the shut down of the domain,  I needed to find a new home for the CCG Toolkit code (Deck Studio, Netrep, RONIN), since I believe it is a great resource for anybody looking to cut their teeth on some code or want an idea for building a similar application suite. Most notably I think the use of separate libraries in a SQLite format, plus the potential use of the EAV database model to add new CCGs into the system has a lot of merit.

That said, I’ve gotten most of the files organized and transfered over to CodePlex. I hope a few programmers might be able to stumble across the code too and benefit from it. I still need to trip up the main project zip for Deck Studio and the master database SQL file, but I should have that done soon.

Update: Thanks to Mickey’s comment below, here’s the direct link to the “binaries” or the “installer” version of the apps. No need to worry about source code.

Netrep Archive: Deck Studio / RONIN Installers now Available

A quick note for any of you still looking for Netrep/Deck Studio/RONIN details…

A little while back I went ahead and took down the CCG Toolkit site, since there was no more news and updated being pushed out.

I received an e-mail [thanks Tyler] looking for the latest version of Deck Studio and I realized I didn’t put the installers out on the archive page for download by anybody interested.

You can now get the Deck Studio 1.9 and RONIN 1.7.3b installers on the Netrep archive site. Just in case you’re still using Deck Studio on the side or are interested in seeing how the apps run without having to compile the code. I also tossed in some Sheep and Ojama tokens I created a LONG time ago, they were rather fun.

The site for all of this is http://ccgtoolkit.codeplex.com. Feel free to drop me a line if you have any questions.

Enjoy all!

Deck Studio: Postmortem

For those if you that aren’t in the software developer lingo, the postmortem typically occurs when a software project has been released into production, or even retired. Considering I have yet to do one, I figured I’d take the instance of the latter and use it to help reflect forward. Of course, I found a nice outline for one on the StackOverflow site and I figured I’d pull a few items.

1. What went wrong?

Streamlining the data entry was a big culprit. It got better over time, but initially, especially after I had inherited RONIN and netrep, having three seperate databases, using different technologies, for the same data, really made things rough. The technical divide just in handling this almost stopped the suite of apps early on.

There could have been a better object oriented design, especially with the creation of controls and some common libraries (which happened during the start of the 2.0 process). Deck Studio was documented well enough that I could find the appropriate support methods when needed, but having it all in a single DSMain.vb file was very bad form.

The transition to netrep 2.0 did not go as favorably as planned. There were some genuinely surprising complaints by the end folks about some of the new layout that I took to heart (most notably the auto-complete feature). Ultimately I should have done more beta testing to draw in public feedback for the new functionality. I don’t think I lost much repeat traffic, but it did put a bit of tarnish on the reputation of the site.

2. What went right?

It took a while to implement, but having a common database for Deck Studio, RONIN, and netrep was crucial. It cut the time of set releases dramatically and using SQLite to port things out for the “public consumption” allowed for a strong, normalized database for storage and a fast, denormalized database for reading records.

In addition, getting the public involved through set templates and the forums went a long way to keep everything “duelist oriented.”

Keeping a “separation of concerns” was always a debated, but good decision. While there was a small rulings module added at the end to Deck Studio, the “heavy lifting” rulings application was always RONIN, and would stay that way. There are a handful of “elite duelists” that are looking intensely at the rules and the deck build, but the apps themselves were lightweight enough that you could be running both just fine and have each need met. I still waver back and forth on this issue, but I think ultimately individual apps with powerful extensions would be the way to go.

3. What would you do differently if you ran the same project again?

I would have started the main database to be more “portable” from the beginning. The card data, and the restriction lists, and the rulings, were all tied into a single file, which didn’t make much sense when doing a simple restriction list update. Having this kind of structure would have made things more amenable to plug-ins and modifications, which became more and more in demand as the apps continued on in their life. Optimizing the search user interface would be another thing to modify. I went around and around on this issue with the best way of presenting, and executing, a method of doing granular search on cards (“all Fiend cards with less than 300 ATK and contain “brain” in their names” type things). There were two main methods to the search (inclusive vs exclusive) and ironically enough RONIN and Deck Studio both took opposite approaches with mixed results on both ends.

4. What hindered your progress during the project?

Documentation. Probably the number one killer of any collaborative app, but I spent a lot of time trying to figure out what my own convoluted code did a year later when making a critical update!

5. What have you learned from the project?

The question should be more of what didn’t I learn from the project. This is where I get a little sentimental in a geekish manner and reveal how much I think coding is freaktastic. The CCG Toolkit (and Deck Studio primarily) was the project that I really got to sink my teeth into as a “real” programming project. I was only a couple of years out of college, and I had a little under my belt with line-of-business type apps, so I want’ completely green to the environment.

But by creating tool used by hobbyists and enthusiasts around the globe, I was able to tackle such a wide variety of issues through development:

 

  • User Interface Design (I had never done WinForms apps up until this point)
  • User Feedback (which was taken very seriously in the design of the program)
  • Creating portable storage mechanisms.
  • Creating remotely updating portable storage mechanisms.
  • Repository synchronization.
  • …and a heck of a lot more

 

This doesn’t even begin to get into all the new code libraries I wrote from scratch, and learning about some really cool coding techniques along the way from my mentor/fellow programmer. I also got to see how the .NET Framework was growing, such as how the List<T> class made things TONS easier for keeping sub decks together. It was genuinely a joy to sit down, sometimes on the weekend, sometimes late at night, and pound out that little update or set release that made somebody else on the other side of the globe happy. Call it a “perfect storm” type application.

Ultimately the toolkit revolved around the data, sorting, viewing, saving, organizing. So now I got to work with three (and almost 4 or 5) different “faces” to the same bit of data. Netrep was the web front end. RONIN was the client based version for the rules. Deck Studio was the client based version for organizing things personally. At one point there was a web based version of Deck Studio (horribly bad, but fixable) and there was thoughts of a mobile optimized version of netrep for viewing things. What a great foundation to get your feet wet with things.

Let’s not forget all the awesome folks I met “digitally” through the process. My mentor was in Balitmore, another guy in Pittsburg, then I’ve had consistent data submissions by some folks in England, Italy, and most recently South Africa. Feedback on the application came from most places, and I probably couldn’t track them all, even on my internal tracking application. The feedback was always constructive and helpful. I cannot thank those folks out there (djp952, Maruno, FloWen, Deathjester, YugiDad, all come to mind, but there are plenty of others that I apologize for forgetting here) for what they did!

But alas the time had come where more important priorities and the volume of work involved dictated that I need to cut my ties and start a new chapter. It is bittersweet. The code and data is up so that others can take the torch and run. Somebody is already in the process of taking over netrep, which makes me happy. There are new apps to write down the road, but you never forget your first.

The Toolkit Backstory: Part 3

As I had mentioned at the end of my last post, things were not faring too well on the Empyre Group front. Empyre Group was closing its doors. I never found out the full reasons behind things, but I do know there was intent to make money wtih the various ventures Empyre Group was putting forward, but they were not panning out. In the wake of this, all of the entities within Empyre Group (City of Gamers Forums, netrep, Deck Studio, RONIN) were given to the various members that worked on them to continue with them as they saw fit. This wasn’t too big of a change, since we were a pretty close knit group already, sharing the tournament card registry and providing support through the forums.

Effectively we had all be doing this in a “hobby” format, donating our free time and resources to the cause, and there comes a point when you need to get off the bus, for whatever reason. djp952, my “Yoda” through this process, was the first. RONIN really had been the flagship program for things, so I was sad to see him go, but didn’t blame him one bit. In the wake of his leaving, he offered up all the code to me to keep RONIN going.  I was honored that he’d consider me for the job and I took it. The netrep/CoG guy was generating the tournament card registry files, so he’d send me updates and I would generate the appropriate libraries and push them out to the respective programs.

Then about 6 months later, daamazing1, another Jedi for the cause, was getting ready to leave the fold too. CoG had already been given off to somebody to maintain (the amazing Digitial Jedi, still going strong with the venture) and I was given the opportunity to take over the netrep site if I wanted. This was a difficult decision for me. On one hand, I was struggling to keep RONIN and Deck Studio up to par with some code changes and some set change. Since both applications were using the exact same data, it made sense to update them to look into a common directory for this information. Not to mention the features updates I wanted to start adding to both programs. I would also need to do web site maintenance on top of this. On the flip side, all three applications had really become a “suite” of sorts and the website was very popular. In addition, netrep was the main source for the data, so I’d have to come up with my own scheme for the data and come up with it fast. So I decided to keep netrep. I could easily keep it running and update things when the need presented.

So here I was, all of the sudden in charge of three major, production grade applications, all using the same data, but in different forms, and I needed to keep them up and running with the latest sets, that seemed to come out about every month. And then the pressure hit. During one of the new set releases, a new monster “subtype” came out called Gemini Monsters. The way the current database was modeled, all this implied was that a new column called “IsGemini” be added to the database structure and you apply the appropriate values. The problem with this was that this also required changes to the structure of the “portable” TCR used by Deck Studio and RONIN. To make matters worse, the object class that stored card data for Deck Studio decks was designed to be “independent” and contain all of that information as well, but as an object attribute. Due to the rigid format of the binary serialization process with custom objects, even adding a simple parameter would break all decks up to this point.

This raised the big red flag that a new database structure needed to be implemented. This also raised the flag that a new deck format was needed to store the decks in. I had brainstormed some really cool ideas with djp952 back in the day about what could be done, and now was the added motiviation to make this happen. I had been fortunate that Gemini monsters weren’t too popular in the game format, but later on, Synchro monsters hit the scene and that made things even worse due to their popularity. As a matter of fact, I’m still in a hole on this one and need to update some images of synchro monsters accordingly.

So that leads us to now, or at least the to October of last year, when the “retooling” started. New database, new deck, new tool names, and while I was at it, I should start expanding into multiple CCGs, since there are duelists out there that play multiple CCGs and having a single app to handle this stuff would be pretty slick.

The Toolkit Backstory: Part 2

 So I was in! I was part of a group making some good software and overall improving the Yu-Gi-Oh! community. To my suprise, there was already a decent infrastructure in place. There was the netrep site, quickly moving up the ranks since it had card data and rulings. There was RONIN, being quickly consumed by judges everywhere, and the City of Gamers website, which was kind of the hub for the applications, but more importantly was standing out as a forum site that didn’t tolerate the flaming and trolling that a lot of other sites were known for.

 

So my task was to become a giant like the rest of the ones already in place. I was intimidated, to say the least [shoot, I'm still getting my feet wet in any kind of intensive app outside of CRUD stuff]. However, the lead RONIN guy took me under his wing and gave me lots of advice when I needed it and showed me how to link into the existing RONIN database to solve one piece of the puzzle [Forever my Yoda, you ROCK djp952].

 

From the beginning, I always wanted Deck Studio to be a “by duelists for duelists” type program. Since I had recently converted to the fiendish arts in my own playstyle, and barely any cards to support it, this was the perfect time to eat my own dogfood. The first thing on my mind was the cards themselves. I’m somewhat of a data junkie anyway, so being able to see a complete list of cards was simply golden. In addition, there was so much good artwork going on to these cards that it was a must to have it available for the duelists. Even now I like to take a peek at Crush Card Virus and dream of the day I’ll have an actual card in my hands.

The wish list then began to flow of all the things that needed to go into the application. The entire deck contents needed to be viewed, but being able to drill down into your monster lineup to verify you didn’t have too many tribute monsters was also needed. Spells and Traps would also need to have this kind of drill down ability. In my own deck building, I started to track what kind of strengths and weaknesses the deck had. I’m also a technical and combo-style duelist, so I needed a place to be able to track all the combos the deck posessed. I’m well aware that a lot of folks probably wouldn’t want to track this data, but it was easier to have it there and not use it, than put it in later.

 

It was a given that the application should validate against the current ban lists. One of the goals of Deck Studio was to allow duelists to scrutinize their deck the day before a big tourney, trying out various incarnations, without having to constantly be pulling cards in and out of their tins/binders/whatever until the final deck was polished and ready to roll. This is also what led into the deck printing feature. I never did find out how much that feature was actually used, outside of a judge who was using it for printing deck lists at tournaments.

 

Organizing all of this information into a layout that comprehensive, but not obtrusive, proved to be the biggest challenge. Part of the reason “Studio” exists in the name of the program is that I was inspired by the layout Visual Studio uses. By having the tools and properties available on the “edges” of the workspace, it allowed you to focus on the deck itself (in the center) and easily drag in the components as you needed them. If you were really focusing on deck layout, or didn’t have enough space, you could simply hide the panels you don’t need, or slide them out temporarily to drag items over. Similarly, items like the deck validation and simulation were important pieces to the application, but the average user wasn’t using them all the time, since the primary activity was building the deck. Having a slide up “status” window worked perfectly since it was akin to the “build” process that occurs when compiling the source code in Visual Studio.

 

So all of this thought, coding, messing up [extensively], and testing led us through several beta cycles [named aptly after fiends of course] until Deck Studio went to 1.0 status. As satisfying and exciting as the 1.0 release date was, it was only a quick stop off. There were still so many features that didn’t make it into 1.0 and there were now plenty of small bugs and support features that needed to go into things. Maintenance life had begun, which was a mixed blessing. It was great to see that there was enough use of Deck Studio that people were finding bugs, but a little tedious in that it prevented me from being able to work on feature enhancements.

 

Sadly, around the time Deck Studio was at 1.5 status, Empyre Group was not faring too well, and I’d be spending the next year or so making some big decisions on my own and having to divert feature enhancements even more. That we’ll save for  the final installment of the back story.

 

:: Peace out homeskillets! ::

The Toolkit Backstory: Part 1

Back in 2004 or so, I was just getting my feet wet in the Yu-Gi-Oh community. I loved the game, but was a little bit of a “closet duelist” since I had no clue how many 20 somethings were actually playing the game. My excuse was to get into judging. I figured that way I could get an idea for the field, plus getting a handle on the game and helping others do so was a big passion of mine.

Through this process I stumbled upon a program called RONIN, the Rulings Online Network Information Node. This tool was simply beautiful. It had all of the current official rulings and an easy method to retrieve them. The complete index of cards was listed on the side and there was an auto-complete textbox attached to it to quickly get you to the card you needed. On top of that, there was an advanced search that allowed you to narrow down specific types of cards to search.

What was really going on here was that I now had a rulings tool AND a basic card searching tool when I wanted to build my decks. I had not the time (nor the finances) to track down every card in the game (which was still pretty limited mind you, this was the second or third set released). This excited me to no end.

In addition to the functionality, my programmer site was giddy over the architecture of the program. There was the online component to viewing data (over a web service), an offiline mode which consisted in generating a data module to store all the rulings, a separate module that stored all the card names that was dynamically. A whole slew of stuff I was itching to try out since my primary job is in the web application side of things. On top of that, it was written in .NET, something I was getting immersed in and run really well, despite a lot of hate it seems to garner.

So I jumped over to the City of Gamers (CoG) forums, which was the host of the application. I was hoping to start pinging the brains of the folks over there for info, and maybe even help join the team to further the cause. I had some really warm feedback from the lead (and only for that matter) developer on things and I make sure to hang around, offer some feature requests, attempt to debug issues, and the like, just waiting for the chance to jump.

…and then it happened.

Empyre Group, the folks running CoG and RONIN, were offering up a programming contest to help them add to the functionality of RONIN. More specifically, they wanted a tool that could be run daily that would ping the primarily rulings site to see if there were any additions, modifications, or removals to rulings, and generate a report as such.

[This is it! My big break! Don't blow this one!]

So I spent the next week coding like a madman. This was all new territory to me: .NET 1.1 Framework (I was still working my way in from a Perl/PHP background), WinForms, HTML Scraping using Regular Expressions, some form of file storage for this stuff. Everything!!! I had no clue how many others were entering this contest, but the traffic was increasing on the CoG site, and I’m sure that there were other folks of my ilk playing the game, and I was still a noob, so I had to impress.

In the end, I was rather impressed with the results, especially for my first shot at things. Don’t get me wrong, the overall package was still pretty ugly. I discovered that DataBinding to DataGrids in WinForms was a bit more complicated that in WebForms, I had part of the application configuration through a menu option in the application, and another part of it stored in the app.config file, and a few really ugly strings that had to be used to the regular expression split on the card and ruling dividers since I hadn’t optimized that process yet. But I had a nice little acronym for it RUN (RONIN Updating Node). Yes, I am an acronym junkie, you’ll have to get used to it. Even more cleverly [and probably to my demise], I could say “Run RONIN Run!!!” and let the application update itself.

I submitted the application and waited, agonizing for a few days. The source code was to be provided too, which would either help or hinder, since my coding style was still quite green and not too nice. To my credit, I did have commenting drilled into my system pretty early in the game, so that should score a few points on make sure things were properly outlined.

A few days later, I received an e-mail from the lead developer at Empyre Group. [GULP] Pausing a moment before opening the message [Did they hate it?! Did the app actually run when they fired it off?! Was the code abysmal?!], I opened up the message and discovered I HAD WON! One particular phrase stuck out in the message… the application was a bit of overkill from what we were expecting… and I never found out if that was a good or a bad thing. I did discover later on that I had been the ONLY applicant to the process. 8^D However, what was more important was that A) I was getting some Yugi swag as the primary prize, and B) they wanted to know if I was interested in joining the team on a new project.

[Are you kidding?! That's why I entered in the first place!!!]

With only a little hesitation, I said most definitely I wanted to join. I had to clear things with my wife first, because from what I was gathering, Empyre Group was getting ready to move into high gear and that would mean some late nights slingin some code. She approved, as long as EVERY night wasn’t a coding night. From there the proposition came to me. Empyre Group wanted to have a deck building program that would better fill the needs of the deck builders out there, since RONIN wasn’t designed for that. The best part was that they wanted ME [Really me?! The rookie in the crew?!] to come up with some ideas. They weren’t pushing anything on me, at least initially, they wanted to know my thoughts on the issue. [Really me?! The rookie in the crew?!]

So I began to start learning a TONS about coding from my own coding Yoda, djp952 (mad props to ya, if you’re still out there peeking in every now and then!). A lot of it came in the form of viewing the RONIN code and discussing ideas on how this new deck building program would work. So thus began my life as an EGI coder on the side.

In the next issue…the evolution of Deck Studio.