Thursday, 31 December 2015

Stop recycling game objects

Many consider object pools a staple of game development. Here's a few examples:

Although Wikipedia does hint at potential pitfalls, there is a much greater problem than having (or forgetting) to reset pooled objects: how do you ensure that references to your objects are cleared before pooling them?

Your object may have registered to receive events.
Or it may be added to a list somewhere.
Or it may be assigned to a field in about a couple dozen places
...
Yes, there are possibly hundreds of objects in your runtime which may hold references to an object you'd like to recycle.

In languages like C++, it is your responsibility to clear these references. Not clearing references before destroying an object leads to wild pointers. Wild pointers lead to intermittent crashes and bugs.

In C#, Unwanted references to pooled objects fail subtly: your application won't crash but a (usually unknown) number of objects will continue to talk to your object without knowing that it's been recycled.

Pooling is an optimisation technique. If you have to use it, know that just resetting these objects is the easy part.

Tuesday, 29 December 2015

Visualising C++ #1 - A quick look at Visual Paradigm

I've been wanting to visualise one of the C++ projects I'm working on for a while. First I tried Visual Paradigm

Surprisingly, VP choked on irrelevant files ( git meta, makefiles, ... ). At the time of writing (Dec. 2015) their help-desk confirmed practical limitations in this area.

Anyways. I somehow got rid of the gits then processed caffe through the so-called 'instant reverse' feature. Selected all output and generated a diagram. Output didn't look great. Here is a second attempt displaying only the 'caffe' namespace:



Not excluding the possibility that this can be configured to produce decent output.

Friday, 30 October 2015

Water in Unity

While fantastic at a glance (well, almost...) water in Unity can be tricky to setup. Water and Pro Water aren't too difficult to figure. Water 4 may not always reward your patience.

Unity ships with 3 water assets

  • Simple water does not provide reflection or reflection
  • ProWater supports reflective and refractive water (foam can be added)
  • Water 4 supports reflective and refractive water with foam and mesh animation.
Compared with Pro Water, Water 4 looks "later and better". 

Foam with Pro Water

  • There's a modified script here. Adds an interesting foam effect; works somewhat differently from foam in Water 4 (tested).
  • Modified water here (inc. underwater)
  • Shoreline generator
More about Water 4

Where does the color come from?

Base color, mostly. However, if you zero Fresnel power, colour is mostly the reflection colour.

What controls the foam?

Tip: Replace the foam texture with something obvious to get a better idea of what's going on.

Under 'Auto-blend', edge controls blending of water with land. High values (1 or more) make foam more visible. To make the foam ring thinner, use high values (1 or more) of shore whereas foam will be very spread out if your values are something like 0.5 or less.

Bump tiling decides how big the foam texture is tiling. Looks like they are tiling two layers of foam too. 

Foam intensity can help to make foam more visible. Above 1, tends to whiten everything.

An annoyance is that 'foam' is also used everywhere on the water surface. feels like a kind of rippling reflection. Disabling Gerstner displace can help.

Thursday, 15 October 2015

Caffe on Win64: the Appveyor setup

In an ideal world, this is how we would build caffe with Appveyor:

md build
cd build
cmake ..
cmake --build . --config Debug
cmake --build . --config Release

Upon running this we find that a few things need fixing.

CMP0054, 0046 policy warnings

This warning is caused by a newer version of cmake. And this is how we fix it (at the top of CMakeLists.txt)

if(POLICY CMP0054)
  cmake_policy(SET CMP0054 OLD)
endif()

Boost libraries

Appveyor includes the following:

 C:\Libraries\boost\stage 
= > sample lib: libboost_date_time-vc120-mt-1_56.lib

C:\Libraries\boost_1_58_0\lib64-msvc-12.0 (or 1_59)
 => sample lib boost_date_time-vc120-mt-1_58.lib or dll

I target the local build (boost/stage) with the following cache entries:

BOOST_ROOT=C:/Libraries/boost
Boost_USE_STATIC_LIBS=ON
Boost_Compiler=-vc120

USE_STATIC_LIBS reflects the 'lib' prefix; The compiler setting is needed because I do not target a specific compiler for building caffe, causing CMake to default to 140 (latest installed on Appveyor) whereas boost libs are built with 120.
  
Protobuf

We download and build from source using the following script:

git clone -q https://www.github.com/google/protobuf protobuf
cd protobuf/cmake
md build
cd build
cmake .. -DBUILD_TESTING=OFF
cmake --build . --config Debug
cmake --build . --config Release
cd ..\..\..


The following cache entries are used in the Caffe build:

PROTOBUF_LIBRARY="../protobuf/build-output/Release/libprotobuf.lib" PROTOBUF_INCLUDE_DIR="../protobuf/src" PROTOBUF_PROTOC_EXECUTABLE="../protobuf/cmake/build/Release/protoc.exe"

Alternative: using releases? (in progress)

Currently there are several release and pre-release versions of protobuf (here). The latest release (2.6.1) does not appear to include win64 binaries.

HDF5 (1.30pm)

An *.msi installer is available. We download and install using the following script:

#download HDF5
$file = join-path (pwd) \hdf5.zip
$url = "http://www.hdfgroup.org/ftp/HDF5/current/bin/windows/hdf5-1.8.15-patch1-win64-vs2013-shared.zip"
(new-object net.webclient).downloadFile($url,$file)
#unzip archive
Add-Type -AssemblyName System.IO.Compression.FileSystem
$out = Join-Path (pwd) \hdf5_unzipped
[System.IO.Compression.ZipFile]::ExtractToDirectory($file, $out)
#install HDF5
cd hdf5_unzipped/hdf5
Start-Process "HDF5-1.8.15-win64.msi" /qn -Wait
cd ../..

Configuration

The HDF group provides a little information. FindHDF5 will leverage an environment variable (not a cache entry) set as follow:

$env:HDF5_ROOT = "C:\Program Files\HDF_Group\HDF5\1.8.15"

LMDB, LevelDB, OpenCV and CUDA

Since we don't use any of the above, turn them off.

ATLAS

We use OpenBLAS. Appveyor  build works already have MinGW (OpenBLAS dependency) so we need not install it.

Download and install

#download OpenBLAS binaries
$file = join-path (pwd) \OpenBlas.zip
$url = "http://sourceforge.net/projects/openblas/files/v0.2.14/OpenBLAS-v0.2.14-Win64-int64.zip/download"
(new-object net.webclient).downloadFile($url,$file)
#unzip OpenBLAS binaries
Add-Type -AssemblyName System.IO.Compression.FileSystem
$out = Join-Path (pwd) \unzipped
[System.IO.Compression.ZipFile]::ExtractToDirectory($file, $out)

Cache entries

BLAS="open"
OpenBLAS_INCLUDE_DIR="../unzipped/OpenBLAS-v0.2.14-Win64-int64/include" 
OpenBLAS_LIB="../unzipped/OpenBLAS-v0.2.14-Win64-int64/lib/libopenblas.dll.a"

Python

If we don't care about Python the following cache entries are used to speed things up and doesn't require numpy:

BUILD_python=OFF
BUILD_python_layer=OFF 

"Cuda is disabled" warning

When CPU_ONLY is enabled, we should not received a warning or error. Issue a STATUS message instead.

GLog and Gflags

GFlags and GLog appear to automatically download and build as part of a caffe build. However, this causes errors.

With GFlags (v2.1.2) we see an incompatibility with recent versions of VS (fixed on master).

error C2084: function 'int snprintf(char *const ,const size_t,const char *const ,...)' already has a body [C:\projects\caffe-cx7ht\build\ext
ernal\gflags-prefix\src\gflags-build\gflags-static.vcxproj] [C:\projects\caffe-cx7ht\build\gflags.vcxproj]
C:\Program Files (x86)\Windows Kits\10\Include\10.0.10150.0\ucrt\stdio.h(1932): note: see previous definition of 'snprintf'

Looking into gflags.cmake we see that we are downloading the latest tagged version (2.1.2).
One way to solve the above error is to downgrade the generator (-G) to Visual Studio 12 2013.

Like GFlags, Glog (v0.3.4) downloads and builds automatically. Building on Windows,  error:

Performing configure step for 'glog'
CMake Error at C:/projects/caffe-cx7ht/build/external/glog-prefix/src/glog-stamp/glog-configure-Debug.cmake:16 (message):
Command failed: 77
'env' 'CFLAGS= /DWIN32 /D_WINDOWS /W3' 'CXXFLAGS= /DWIN32 /D_WINDOWS /W3 /GR /EHsc' 'C:/projects/caffe-cx7ht/build/external/glog-prefix/src/glog/configure' '--prefix=C:/projects/caffe-cx7ht/build/external/glog-install' '--en
able-shared=no' '--enable-static=yes' '--with-gflags=C:/projects/caffe-cx7ht/build/external/gflags-install/lib/..'

This with a redirection to C:/projects/caffe-cx7ht/build/external/glog-prefix/src/glog-stamp/glog-configure-*.log where we find that CMake... didn't find the C compiler?

Looking into this, we find that caffe includes two CMake files used to download and install GLog and GFlags. See glog.cmake.

The DIY route

In our current setup we build Glog ourselves.  Because we do not build GFlags, on which there is a dependency, this causes a linkage error:

LINK : fatal error LNK1181: cannot open input file 'C:\projects\caffe-cx7ht\gflags\build\lib\Release\gflags.lib'

Even so, we have satisfied enough dependencies that CMake agrees to attempt a caffe build.

Appveyor notes

After a build we can ask Appveyor to wait so we can use remote desktop to diagnose issues:

$blockRdp = $true; iex ((new-object net.webclient).DownloadString('https://raw.githubusercontent.com/appveyor/ci/master/scripts/enable-rdp.ps1'))

YML setup

Appveyor gives the option to drive build works either using a script or via a GUI. We prefer to save the YML. Powershell scripts are defined separately because embedding PS within YML is somewhat inconvenient.

Caching

Caching build results is used to help speeding up builds. Currently we cache Protobuf and GLog. See here.

Wednesday, 7 October 2015

10 minutes with Unity remote

Do you think waiting 2-5 minutes to get your build on a real device is too much? 
Unity remote is for you and setting up is child's play.

Unity remote ferries video and input between your dev box and an iPhone or Android. The game processes and renders on your dev machine but you can preview (and interact with) the result on-device.

While testing reported frames rate dipped from 50 to 25fps. Somewhat unresponsive, therefore less than ideal for testing action gameplay (tested on iPhone 5 and a 2010 Macbook Air).

Additionally, while you might expect your game to behave the same as it would on device, inputs are actually enabled from all devices (editor and your handheld), this may be error prone.
  • Avoid code relying on a keyboard or mouse NOT being present.
  • Avoid querying device type. While using the remote, device type, name and model (as reported by SystemInfo) reflects your dev machine, not the target device.
  • Input.touchPresent returns false while using the remote.

Thursday, 24 September 2015

Blender and Mixamo

While the future of Mixamo is uncertain, at the time of writing (September 2015), there is a working integration with Blender.

Importing

Assuming you have a rigged, animated character created using Mixamo you export to Blender via FBX.
Upon importing the FBX, you may wish to change the model's scale. Sometimes models appear very small inside blender.

Issues

The main known issue ( At the time of writing! ) is that the model's rest position is not compatible with that used by animations. Pretty broken, eh?
Although it takes a bit of patience, you can pose your model to look like the animated rig's rest position, then apply the armature modifier in Blender. In effect this modifies the geometry of the model to make it compatible with the rig.

Limitations

  • Keys at every frame. Take patience to remove unneeded key-frames (may reduce file size). 
  • No inverse kinematics. Inverse kinematics can be added to the rig but this will not retroactively apply to imported animations.
  • Non standard bone names. This is unpleasant (long prefixes) and will break mirroring schemes. I wrote a small script to fix this.
  • Inverted hip bone. The solution to this is to uncheck 'inherit rotation' from the spine bone. Works, but animations created in this way should be kept separate from your Mixamo animations.
  • Unexpected rotation of some bones. With leaf bones (e.g. fingers, head) as far as I could judge you can rotate the bone's rest position without breaking existing animations.
Creating new animations

Given the above, you can create new animations based on the Mixamo rig. Depending on how your rig is setup you may need to create these separately from your Mixamo animations.

Remember that Unity doesn't need all animations to be stored in the same file. So creating your own animations separately won't be a problem. If, on the other hand, you don't know how to put several animations in the same file, read here.

Monday, 31 August 2015

Experimenting with friction in Unity

Initial setup

Let's create a plane. Next create a cube and place it right on top. Add a box collider and rigid body to the cube.

Basic friction

Let's create a Push script adding a force to the cube at every fixed update:

GetComponent<Rigidbody>().AddForce(force,ForceMode.Force);

ForceMode.Force indicate a continuous force. The result of applying a horizontal force is interesting: under 10 units (Newtons, I hope) the box doesn't budge. At 10, it doesn't slide, instead, it topples. This strongly suggests that friction is present.

Next we create a so called Physic Material. Default friction value is 0.6 for static and dynamic (see definitions here). At around 0.3 the box is moving, and doesn't topple.

Moving platforms

Turn off the Push script and create a Move script applied to the plane. Since the plane is not a rigid body no point in applying forces. Instead we generate a linear translation inside Update() or FixedUpdate()

transform.position += velocity * Time.deltaTime;

The plane moves, but the box doesn't. So we have friction, but it's a one way affair. Yet the box and plane are still interacting: Once the plane has been removed from under it, the box falls (though, it doesn't topple).

Add a rigid body to the plane and check kinematic. Restart the simulation. Now the plane still slides under the box. Upon reaching the edge, however, the box topples.

Still no friction?

Uncheck kinematic and gravity. This time we get an error: Unity considers that the mesh collider is not convex so it won't allow physics on it. I wouldn't try making the plane convex so I replaced it by a box collider. Without gravity the plane won't fall on its own. However the box is now pushing it downwards. No good.

We don't want this plane to fall so, freeze Y position along with all rotation axis. Now the plane yet again slides under the box, which finally topples.

Still no friction?

Substitute the original Push script to the Move script. Now the plane is dragging the box away (this works with any Force mode).
Finally, re-apply the physic material and lower the friction. Somewhat surprisingly, it now takes significantly low friction values (~0.01) to the box to topple off the plane.

The uncanny experiment

In rigid body settings, freeze the plane's location and rotation (all axis). Run the simulation and observe: the box is moving, then topples off the plane: the plane now behaves like a rolling carpet.

Conclusion
  • Friction is a property of colliders, not rigid bodies.
  • Friction applies to moving rigid bodies using forces: when moving a solid over static terrain, we needn't add a rigid body to the terrain to simulate friction.
  • Friction doesn't apply to kinematic bodies: therefore, to define a moving platform with correct friction:
    • Must have a collider
    • Must have a rigid body
    • Kinematic is unchecked
    • Move the platform using forces.
    • Take advantage of rigid body constraints.

Wednesday, 26 August 2015

Did Mixamo solve the animation problem?

At the time of writing, all commercial services at Mixamo are down following their purchase by Adobe. In the meantime, their animation system is in demo mode. And it looks like Blender support improved. Let's have a look.

A first run with Mixamo

(21:40) I exported my model (obj format) from Blender. I decided to only use the .obj since I don't have textures.

(21:52) Auto-rigging config is manual but placing joints only takes a couple of minutes (really). Then according to a caption, it will take about 2 minutes for auto-rig to complete.

(22:04) Chose a few basic animations. Plays nice in preview, now I would like to export them. Go My Assets and select Collada as export format. Oh well. This proved to be somewhat fiddly as I needed to checkout animations. Although their shop is down this is still doable.

(22:12) Let's try to export these animations. Choosing collada, non uniform frame reduction

(22:37) I could get my model and animations back into blender but the export results looked weird.

(22:49) After fiddling for a bit, I somehow established a correct import configuration.

Tips
  • Correct DAE export for blender requires exporting with skin and no keyframe compression
  • On some animations (typically, walking, running, ...) check 'in place' to prevent the model from moving while animating.
(P)review

Mixamo is a complete animation solution. It seems to work pretty good, especially with humanoid characters.
Auto-rigging isn't a feature that I absolutely need, and I'm not sure how it would fare with complex meshes. Apparently, it works.

Pros:
  • Good performance in web client.
  • Blender import actually works.
  • Free stuff
Cons:
  • Their animation collection is big, not huge. Maybe enough to prototype stuff and create a AA game.
  • High number of keyframes may be affecting file size.
  • In Blender
    • Animations need to be imported separately
    • Rig and model import with each animation.
    • Animations get in with incorrect names.
    • Models get in wrongly scaled
    • Inept rigs
    • One key frame per frame?
Until then...

Adobe purchased Mixamo, pulling the plug on sale services without a go-live date. They also pulled the plug on Mixamo's UAS (Unity Asset Store integration) so it's overall a very bad time to invest time and effort into Mixamo; more so considering that, it's hard to believe that Adobe will encourage improving Blender integration. Casual devs will welcome the fact that some of their tools and services (10 rigs, 20 animations) for free while they transition into the unknown.

About aeronautic coordinates

Often confused with Euler angles, aeronautic coordinates are useful in games (even if you're not writing a flight simulator)

Used in 3D applications, Euler angles 'in general' often become an endless source of surprise, confusion and pain. In contrast, aeronautic coordinates use well defined conventions

Please assume a vehicle's nose (think of a plane) is pointing towards Z positive, and its right wing is lying along the X axis. Many game developers find this orientation intuitive.

Aeronautic coordinates, consist in pitch, yaw and roll, also known as atttidue, heading and bank.
In some litterature you will find the term 'attitude' used to refer aeronautic coordinates altogether.

Unity3D implementation => [source]

Useful definitions.

  • Pitch/Attitude - an angle indicating whether a vehicle is flying upwards or downwards. 
  • Yaw/Heading - an angle indicating a vehicle's general direction/course. In the real world, cardinal points (north, south, east, west) are often used to refer a vehicle's heading but this is unpractical when dealing with computers.
  • Roll/bank - an angle indicating whether a vehicle is tilted or not. For example, a plane is 'tilted' when their wings do not lie in the horizontal plane. 

Calculations

While there are many ways to extract aeronautic coordinates from a transform, and quite possibly several correct ways to do so, I will suggest an approach which I hope you will find safe and intuitive. 

I assume that you have access to the forward, right and up vectors associated with your transform.

Then you can use the following recipes:

Pitch:
   Return -asin( forward.y )

Yaw:
   Let F be the ground projection of the forward vector.
   Let R be the ground  projection of the right vector
   If u is small/degenerate:
      Let α be the angle between R and the right vector (1,0,0)
      Return R.z<0 ? α : -α
   Otherwise... 
      Let α be the angle between u and the forward vector (0,0,1)
      Return F.x>0 ? α :-α

Roll
   Return -asin(right.y)

Important: Your forward, right and up vectors must be normalized.
Occasionally APIs return components (slightly) without the [ -1, 1 ] inclusive range. clamp right.y, forward.y to make sure that this doesn't happen, otherwise asin would return incorrect values.

Additional note

With some implementations, the heading is undefined when a vehicle is pointing straight up. This satisfies the (flawed) intuition that a vehicle pointing upwards isn't going north, east, west or south. It is a flawed intuition because, even when a plane is flying straight up, you can infer its heading from the position of its wings. The suggested recipe and linked implementation leverage this idea to avoid errors.

Thursday, 6 August 2015

Smartphones still strong against tablets

Unity Technologies happen to maintain detailed statistics suggesting that smartphone users outdo tablets on their install user base. Here's a summary of the iPhone vs iPad breakdown:
  • iPhone: 62%
  • iPad: 37%

Tuesday, 23 June 2015

Bootcamp, a late 2010 Macbook Air, Win 8 Pro, an SD card, and a SL of BS

Bootcamp does not agree to my custom, partitioned drive.
Depending on the target version of windows, running from an SD card may be possible (see Vanderi's blog).

Backing up everything.

Using disk utility, I un-partitionned without re-installing OS-X (Select the HD, go to the 'partition' section then remove all but the boot partition and recover unused space).

Bootcamp is asking for an optical drive. I want to install windows from my SD card. This is only a warning. Ignoring it (for now).

Next it's complaining that I don't have USB drive plugged in. Confusing but, actually, bootcamp is trying to download drivers needed to run windows on my Mac.
Let's get a USB drive.
Needs to be FAT formatted.

Bootcamp tells me that it can download drivers for Windows 7. Pity, because I don't have that, which MS aren't selling anymore. No support, no problem (says Ars Technica)? Apple's compatibility chart insists that I can't.

Success story installing Win7 then upgrading.

Bootcamp is being pissy about my Windows install files sitting inside an SD card. Apple confirm it's not okay.

Maybe I can install windows 8 without Bootcamp. Software 7 blogged about it.

Windows wants NTFS but all we can make is a FAT partition, which is fine.
Back to Disk Utility let's make a FAT partition.
I now have a FAT partition, Wakugumi.

Looks like I might get a chance to install Win 8, if I hold down the option key at restart, and should expect a bumpy ride.

Nice, except it doesn't work. Proposed solution. Oh wait. Look here. And... well. After trying everything else, what worked for me was simply to make some "free space" instead of creating an actual FAT partition with Disk Utilities. Then start installing windows, and create the partition using the provided interface.

Software 7 suggests a patch to improve stability; I didn't wait to verify whether it was stable or not, I applied the patch.

Software 7 signal other issues; to mitigate this I tried running my Bootcamp drivers. The top exe refuses to run outright but I was able to run a batch of semi-randomly selected drivers, which somewhat improved the behaviour of the track-pad. Apparently it used to be possible to download the Win 8.x drivers without fuss (meaning, from Apple), no longer. Got them from a free DL site. Still waiting. Oh wait... can still download them from Apple.
  • Track-pad not working. Works better now but still no 2 fingers scroll.
  • Can't adjust volume and brightness. Not solved. My display is bright, and there is no sound.
Downloaded 8.1 Bootcamp drivers. Driver won't be fooled, still doesn't run. I can selectively install device drivers.

Upon restarting, Windows indicates that the 'PC' needs repairing, undoes all my changes, reruns minor installation cores.

From there...
  • Using BootCamp5.1.5621 as downloaded from Apple
  • Installed AppleMultiTouchTrackPadInstaller64 otherwise trackpad is really sluggish. Even after installing this, you still have half of a trackpad running with no scrolling gesture, no right button. Suggestions here.
Searching, I found useful suggestions:
  • Bypass bootcamp's compatibility check
  • Run bootcamp in Windows 7 compatibility mode
  • Run with enough privileges.
Bypassing compatibility checks and running with sufficient privileges is explained here and here's a little more details:
  • Don't run via 'Setup'. Instead, run via BootCamp\Drivers\Apple\Bootcamp (depending on your package this may be called BootCamp64 or similar)
  • You can disable the compatibility check by running "trouble shoot compatibility" in BootCamp context menu.
  • Run the suggested command via cmd.exe.
I tried this with drivers downloaded from apple, and I tried this with the Windows 7 drivers downloaded via bootcamp. A clean re-run appeared to work fine initially. That is, until I accidentally closed the MBA's lid while installing Visual Studio. Following this, the MBA rebooted and Windows went into repair mode.

Sunday, 14 June 2015

MultiSub

In July 2014 (one year ago!) I did a little work on a sub multiplexing application. Its purpose is very simple and practical: allow viewers whose native language differs to share subs on a single screen.

Finding Chinese subs

Finding Chinese subtitles has become difficult since shooter.cn went down (but see here). Downloading subs in a foreign language is a roundabout routine anyway.

  • Often, I need to find the name of the movie in its original language.
  • Go to Wikipedia zh, searching the film's original title. This is to retrieve the Chinese name.
  • I use subhd.com to find Chinese subtitles. At the time of writing it looks like subhd.com doesn't support Safari anymore.
  • There are a few subtitle formats around. I use Jubler to convert from substation alpha (*.ass, *.ssa) to SubRip (*.srt).
  • I often process videos in *.mkv format. One problem with this is that subs are baked into the mkv file. On OS-X I use MKVToolNix via homebrew. Once installed, in terminal (this is a command line tool):

    mkvinfo FILENAME.mkv

    => sift through the output to get the subtitle track index and the encoding (often, *.ass or *.srt)

    mkvextract tracks "FILENAME.mkv" TRACK_INDEX:OUTPUT_FILENAME.xxx

    => self explanatory?

Combining subs

Trivially, my utility parses the sub lists, merges, re-indexes. My player (VLC) manages overlaps gracefully. It would be cleaner to do this while merging the subs, but it's a lot more work.

A user interface

Today I built a trivial user interface for multiuser. Took 2 or 3 hours I guess; it is very, very simple.

Plans

I guess I'll try to submit MultiSub to the mac app store. Since I'm not planning on selling it, I could also make it available on free download sites.


Wednesday, 10 June 2015

Squashing: the destructive workflow (git/github)

A while back I attempted my first contribution to an open source project on github. 21 commits laters I was asked to squash them (my commits) to clean things up.

You can do it the right way. But then again, you might not want to. After all, it's a stupid ass pain: modal, raises non-existent merge conflicts, knows better than you do, never heard of user control, wants you to use vim or emacs.

Why do it?

Information hiding. What's 21 commits for me should be an atomic change for the rest of the team.
Since git doesn't have a concept of private/public commits, or whatever non-destructive solution you might come up with, we are stuck with a revisionist, unsafe, destructive approach to collaboration.

Before we start

At some point you may wish to revert local changes. Sadly "revert local changes" can mean a whole bunch of stuff from git's revisionist perspective. See here.
We are trying to collapse a bunch of commits into a single, unique commit, so, in this case, "revert local changes" maps to:

git fetch
git reset --hard origin/master

This will reset clean to your latest remote.

Hint: if you have been rebasing over and over while working on your branch, the number of commits you want to squash is less than the number of commits on your branch. Check the commit history to distinguish your commits from others'.

Squashing without squashing

I'm assuming that all your changes are already committed. One of the methods explained here shows how to non-interactively 'squash' N commits:

git reset --soft HEAD~N 
git commit -m "shiny new commit"
git push -f

Where "N" is the number of commits to squash. Done 21 commits, want a single commit instead? N = 21.

Tip: calling reset HEAD~1 twice is equivalent to reset HEAD~2.

I gather that...
  • "git reset --soft" rolls back our commit number to 'N before now' but without actually changing local files.
  • "git commit..." commits changes locally, with the specified commit message.
  • "git push -f" (so called force push) tells the server to discard previous changes, replacing them with the latest local commit.
No love lost on the git workflow.

Sunday, 7 June 2015

Do not represent an object's life cycle using enums

Avoid 'summarising' the state of an object using enums.

While it is often convenient to idealise an object's life cycle as a linear sequence of states, such representations are inherently inaccurate and fragile; over time they become a source of bugs.

The OOgtech blog, archived

From 2009 to 2013, I ran a game dev blog covering a variety of topics, from OpenGL to agile game development.
You can still find it using way-back machine.

The OOgtech blog

I will refresh my best articles and put them on this blog.

Tuesday, 2 June 2015

Git and Github: setting up to work on an open source project.

In order to test Git and Github, I created a repository, apqa, featuring a nice vector/math library I wrote a while back.

From a project's owner point of view, committing changes is trivial. However, I am exploring Github from a contributor's point of view.

So, after I created my repository and logged a couple of issues, I logged in as a different user and let's get to work.

Setup: Fork, clone and sync.

[web interface] Fork to create a copy of the target repository on the Github server. Said new copy is linked to your github account.

[web interface] Clone your copy of the repository. This creates a copy inside your local machine.
!!! for this to work, you need to download install github's desktop front end and you should be logged into your user account in both the front end application and the web interface.

[command line] cd to your copy of the repo, then:

 git checkout master
 git remote -v
 git remote add upstream https://github.com/ACME/REPO.git
 git remote -v

And the reply to the latest command should be something like.

 BVLC  https://github.com/ACME/REPO.git (fetch)
 BVLC  https://github.com/ACME/REPO.git (push)
 origin  https://github.com/MY_COMPANY/REPO.git (fetch)
 origin  https://github.com/MY_COMPANY/REPO.git (push)
 upstream https://github.com/BVLC/REPO.git (fetch)

 upstream https://github.com/BVLC/REPO.git (push)

[command line] to update master to the latest changes:

 git fetch upstream
 git checkout master
 git merge upstream/master

References

Monday, 20 April 2015

Agile Project Management for Small Projects

While Agile development Gurus may not recommend cherry picking features, I suggest the use of mini-stories and iterations to help deliver functionality while working on small projects.

Are we doing good?

The key to good project management is that, often, the client see progress and the developer get compensated.

How often? Many small projects move slowly, right? I would argue that, the smaller the project, the faster we should see improvements.

  • For a SME (small or medium enterprise), 240 to 360 man-hours may cover 4 to 8 stories.
  • For a small project with a single programmer working part time, 4 to 12 hours may cover a single mini story.
Not convinced? Please consider the following:
  • Small projects are... smaller. Features should be simpler and easier to implement.
  • Many small projects build on top of existing functionality (think "asset store"). While changes to existing source are time consuming, more often than not such changes are small and do not (should) not impact existing structures.
  • Small stakeholders shouldn't commit over large amounts. No more than you should gamble your house before your mortgage is paid for.

Is something wrong with our project?

The following clearly indicates that something needs fixing:

  1. The software isn't working. If the software isn't working on the client's box, at least it should work on the developer's (who can demonstrate progress by uploading a video)
  2. The developer do not know what they are supposed to do.
  3. The client do not know what the developer are doing.
  4. The client and the developer disagree about the result.
I'm not suggesting that you micro-manage your project. If the developer get paid regularly, and the client can see visible improvements, you are doing just fine. Otherwise, read on.


How to manage a small project?

I will assume that you are working on a small project if you put 8 to 24 man hours per week into development. In short:

  • The developer and the client must agree on deliverables. Agree on 2 to 6 deliverables per 24 hours period.
  • Describe deliverables using stories. A story is a common-sense description of a feature which adds value to the software.
  • If possible, give a heads up to your client when you have completed a story.
user stories



Problems and solutions

Developer: "I need a framework" 

Read elsewhere about Big Design Upfront.
Although you may feel that a framework 

I don't understand what the developer are doing

Explain to your developer that you pay for visible progress. Refuse to pay for technical stuff that you don't understand.
Irrelevant to perceived usefulness, some features are expensive to implement. This stuff should average itself over time. You can appreciate this on a velocity chart.

Velocity Chart

When in doubt, you can ask advice from another developer; if you feel that work is slow, you can also get another developer onboard. Adding a person to a project is less risky than firing your developer outright. After all, they may be struggling with genuinely tricky stuff.

I don't understand what the client want me to do right now

Some clients have a core vision / general objective in mind but may not know how to achieve their goal, or may feel unsure about where to start.

Don't be uptight. Make your own task list and show it to your client. In most cases, small clients feel happy to see the burden of project management lifted off their shoulders.

The client are unwilling to pay until the software is 'complete'

Explain to your client that you expect a compensation for work done, and give them means to monitor progress. If this doesn't satisfy them, you don't have to work for them.

The client will pay after reviewing the code

Do not agree to this. Agree to small deliverables, so your client can review the code after they pay for it.

90% of the software out there, including software used in ATMs, rockets, the power grid and medical equipment,  isn't "good software". 

Conclusion

In this article I shared my experience working on small projects. I hope that this will be useful to you and wish you success in your endeavours.


Saturday, 11 April 2015

Unity Networking: Playing on the server

Having to open two instances whenever testing a networked application is awkward.

Two questions arise when trying to "play on the server":
  1. Can we setup as both client and server within the same instance? The answer is NO. Beyond the negative result, "Client" and "Server" roles are interchangeable for most practical purposes.
  2. Can we RPC from the server to itself? Somewhat surprisingly, the answer is, again NO. More generally it appears that you cannot unicast from a network player to the same network player.
Short of re-implementing all our input interface to invoke server instances locally, what do we do? Reflection will help overcome this hurdle:

private void ClientRequest(string name,params object[] args){

        
if (Network.isServer
            typeof(TargetType).GetMethod(name).Invoke(avatar,args);
        else 
            networkView.RPC (nameupstreamargs);

}

In the above example...
  • TargetType is the type of the object that you wish to invoke.
  • networkView is the network view that you would normally send your RPC from.
  • Remember using System.Reflection.
More powerful than MonoBehaviour.Invoke( name, delay ).

Friday, 10 April 2015

Thursday, 9 April 2015

Removing Logged out players

Here's a couple of tips to help you kill logged out players and make sure they stay dead.

Destroying network instances

There is a counterpart to Network.Instantiate : Network.Destroy. The simple advice is, whenever possible, use Destroy NOT Network.Destroy

Using Destroy() means that you let a client destroy its local copy of a network object in due time, not in the middle of doing something else.

Do not cache network instances (as a rule, do not cache anything until you are truly desperate. And one more thing: don't optimise). This is a recipe for creating intermittent bugs.

When do we know that a player has disconnected?

Any object can process OnPlayerDisconnected(Network player)



They're coming to get you Barbara!

As soon as you get this to work, you will notice that opening new clients causes logged out players to return from hell. This is all thanks to buffered RPCs. What does it mean for an RPC to be "buffered"? Well, it means that any buffered RPCs will hit every client you open, in play order.

This Q&A explains how you can clear buffered calls to Instantiate.


Wednesday, 8 April 2015

Basic inventory system (2D)

Today I will put together a basic 2D inventory system. This session lasted 2 hours (including the writing of this tutorial).

download the unity package.

Resources

First we need artwork. I downloaded a free pack created by 7Soul1.

The inventory will display (or hide) upon pressing the I key. Since I'm only prototyping I'm not very picky about functionality or appearance. All I need to do is display the inventory and perform a default action upon selecting an item.

I will use the Unity UI system.

Inside the editor, I created a canvas object. This is needed to display UI elements.
I notice that graphics cannot be dragged directly on stage. This is an interesting difference between the 3D and 2D workflow.

A simple, clickable button

I created a button and removed the text element. I configure the button.

  • "simple" (not sliced)
  • preserve aspect ratio
  • change the icon. Notice that we need to change the texture type to "sprite" in order to access imported artwork.
I then create a script named "UseItem". This is to test the button we've just created. The script provides a single function, Use:

       public void Use(){
           Debug.Log ("Applying default action.");
      

       }

Via the Button script, we now tie the button's OnClick function to the Use function of the UseItem class. Works.


A row of buttons

We don't know in advance (before runtime, that is) how many items the inventory will contain so I prefab the button as "InventoryItemButton".

I then create a DisplayInventory script. Initially I will attach this to the canvas object. Let's display the same button we just created, but now we're instantiating the button programmatically.

        GameObject e = Instantiate(itemUI,
           Vector3.zero,
              Quaternion.identityas GameObject;

        e.transform.SetParent (transformfalse);


The prefabbed button must be attached to a GameObject field named itemUI for this to work.
Now we can create a row of buttons, like this:

        int spacing = 34;


        for (int i=0i<3i++) {

            GameObject e = Instantiate (itemUI,                                    

              Vector3.zero+Vector3.right*spacing*i,    
                 Quaternion.identityas GameObject;

            e.transform.SetParent (transformfalse);


        }

A data source

In order to populate the inventory with real data, we need a data source. Minimally, the data source can be a list of strings, with each string representing an item. We will then match each string to an icon.

        string[] inventory = new string[]{ "Small healing potion""Sword""Mace" };
     
        for (int i=0i<inventory.Lengthi++) {

            GameObject e = Instantiate (itemUI
                                        Vector3.zero+Vector3.right*spacing*i
                                        Quaternion.identityas GameObject;

            e.GetComponent<Image>().sprite = 
                Resources.Load (inventory[i],typeof(Sprite)) as Sprite;

            e.transform.SetParent (transformfalse);

        }



As you can see, Resources.Load() is used to retrieve the sprites at runtime.

Adding functionality

One problem with the above is that our inventory items don't do anything. As "Strings" they don't have any functionality. What if our inventory items were actual game objects? Then, we would have a simple solution (I will discuss later why it makes sense to model inventory items as game objects)

For now, let's do the following:

  • Create an inventory game object.
  • Add several items (just use empties)
  • To each item, we'll attach a component called Item. In fact I have several subclasses of Item, including Consumable and Equipment.
We then update the function used to create the inventory:

        Transform inventory = GameObject.Find ("Inventory").transform;

        for (int i=0i<inventory.childCounti++) {

            Item item = inventory.GetChild (i).GetComponent<Item>();

            GameObject e = Instantiate (itemUI
                                        Vector3.zero+Vector3.right*spacing*i
                                        Quaternion.identityas GameObject;

            e.GetComponent<Image>().sprite = Resources.Load (item.type,typeof(Sprite)) as Sprite;
            e.GetComponent<UseItem>().item = item;
            e.transform.SetParent (transformfalse);

        }

And the UseItem script is now updated to look like this:

       public class UseItem : MonoBehaviour {

           public Item item;

           public void Use(){ item.Use (); }

       }

Small embellishments.

I reattached the InventoryUI script to a standard UI panel. This takes a little configuring but felt right. 

I also wanted consumable items to disappear after use. This requires rebuilding the UI. Unfortunately, Destroy() does not take effect immediately so that, if we query the UI to rebuild right away, the removed item would still show. I solved this with a coroutine but it felt a bit clunky. A better option is to un-parent the item being destroyed (since we're iterating the children of the Inventory object).

Why do we model inventory items as game objects?

Before they become part of the player's inventory, many items begin their life as game objects.
The question is then, what happens when we collect these objects? We probably want them to disappear from the game world. 
An effective, but inefficient and inelegant solution to the problem is to just move them out of the way, possibly hiding renderable components and disabling colliders. If you use a model/view split in the hierarchy, you can also destroy the view, and all you have left is the "item logic".
Either way, you can preserve the original object, which preserves the functionality of the item.

Monday, 6 April 2015

Unity Asset Watch: Control Objects with EyeX!

Tobi's EyeX SDK is an asset allowing you to control 3D objects by just looking at them. I've seen eye pointer tech over the years but don't remember whether it is supposed to be any use.

Saturday, 4 April 2015

Multiplayer in Unity: Get connected

Introduction

The goal of this article is very simple; I have two computers in our living room and I want to find out whether I can create a Unity executable that will have these two computers communicate.

Setting up a server

The "server" is the instance of our app that can accept incoming connections.

I created a Server class. I initialise the server using Network.InitializeServer.
Used code as provided in API reference.
Added error handling code.
Seems to work.

Setting up a client

The "client" is the instance that will connect to the server.

I created a Client class. I initialise the client using Network.Connect.
There are endless variants of Network.Connect. The one I choose uses an IP and port (but read on; we're not staying with this)

It works. In fact, it works a bit too well because it still initialises successfully when the network is not initialised. Weird, I know.

How to choose the IP and port
  • The 'port' refers a communication channel. As a rule of the thumb don't use anything under 1000 (other applications and OS services may be using these). Of course the port should match between client and server.
  • 'IP' identifies a computer on the network.
    127.0.0.1 refers the local host. In other words this is connecting to the same machine we are running the client from. Often useful, but we hope to do better than that.
So, how to choose the IP? And why we need a master server

The answer is, let's not do this, as you probably do not have a stable IP address.
Not sure what this means? Don't have stable IP.
Didn't setup your internet connection yourself? Don't have stable IP.

From a functional point of view, the master server is a go-between the client and the server. Works like this:
  1. Server initialises.
  2. Server registers with master server
  3. Client fetches server info from master server
  4. Client connects to the server.
So, where is the master server you ask? Well, it's a service provided by Unity Technologies so I guess it is somewhere inside their bunker.

Note that the Unity master server is provided only for testing purposes. They're not actively preventing you from using it in a live game but they don't warrant uptime either.

IMPORTANT: You cannot setup a client AND setup a server AND access the master server within the same instance. It may be possible to run a client and server via localhost (or using stable IPs) but I haven't tried. To understand why it works like this, read here.

Registering to the master server

MasterServer.RegisterHost takes 3 arguments (API):
  • Game Type - this is a unique identifier for your game. You might use something like com.company.gamename just as a convention that's easy to remember.
  • Game Name - this is what most players think of as "room name". Again use whatever you like (there is probably a way to iterate room names but in practice this is not really necessary).
  • Description - frankly, I think this should be optional. Just put something there.
IMPORTANT: Call MasterServer.UnregisterHost before exiting (for example, inside OnDestroy() )

Client side: Fetching the host data via Master Server

The client uses 3 APIs to connect to the master server and retrieve a host:
I'm not sure about the advantage of polling for the host list. If you implement OnMasterServerEvent you will receive a notification when the host list has been received. A big advantage of doing it this way is that polling will keep busy with itself after an empty list has been received.

Then we connect using Network.Connect( HostData host, string password )


Troubleshooting Tips

Before getting into specific errors I encountered, note that it is useful to have a separate script reporting master server events. Said script will implement the following:
  • void OnFailedToConnectToMasterServer(NetworkConnectionError err)
  • void OnMasterServerEvent(MasterServerEvent e)
Why is this useful? Because these events are broadcast. So if you put them both in your client and your server script, every event will be broadcast twice, which adds to the general confusion.

Other methods that can help troubleshooting:
  • Use separate GameObjects that you attach your client/server script to. This way you can disable the one or the other and see what happens. This also provides a quick way to prepare a "server only" or "client only" build.
Common errors, and why they occur:

Error #1: Already connected to the master server, the server probably hasn't cleaned up because of an abrupt disconnection.

May occur as a result of trying to simultaneously start the client and server inside the same player.

Error #2: The connection request to ###.###.#.###:##### failed. Are you sure the server can be connected to?

May occur while running a client and server in the same app. As far as I could judge, Unity does not allow a client to be its own server (and vice versa).

Error #3The remote system is using a password and has refused our connection because we did not set the correct password.

May occur while running a client and server in separate instances, but the client did not set a password.