Welcome, Guest. Please login or register.
Did you miss your activation email?

Login with username, password and session length

 
Advanced search

1369163 Posts in 64312 Topics- by 56302 Members - Latest Member: maknaqurban

November 12, 2019, 11:38:00 PM

Need hosting? Check out Digital Ocean
(more details in this thread)
TIGSource ForumsDeveloperTechnical (Moderator: ThemsAllTook)Unspoofable Leaderboards
Pages: [1] 2 3
Print
Author Topic: Unspoofable Leaderboards  (Read 6987 times)
bluej774
Level 1
*


It smells like wet fur in here!


View Profile
« on: January 27, 2010, 12:46:25 AM »

High scores used to be a thing of pride.  That is, until high score lists went online.  Leaderboards quickly became deceptive due to the relative ease most of them can be abused.  Jerks began posting impossible scores, thereby ruining the high score experience.  This often hurts the game itself as well.

I'd like to start an open discussion on what can be done to bring honesty back to the high score list.  What leaderboards are out there that are hard to spoof?  I understand even Xbox Live games have had problems with spoofing.  I've heard that Mochi Scores is hackable, but I've never seen it in action.

There must be a reliable way to have online high scores.  The only idea I can think of is some type of encryption.  That seems to have a fair bit of overhead just to submit a high score, though.  Another idea I had was similar to how commercial programs have serial number checking.  But we all know how secure that is.  Roll Eyes

In any case, I believe truly secure high scores are a possibility and the answer may lie in an open source library of some sort made specifically for such a purpose.  Anyone have any ideas how this could be accomplished?
Logged
mewse
Level 6
*



View Profile WWW
« Reply #1 on: January 27, 2010, 01:24:27 AM »

For a standard high score table, there are two main ways to post invalid scores to a leaderboard.  (And lots more beside that, but these two are by far the most common)

First, you have the "spoof", where the attacker writes a program that will communicate with the game server and pretend to be a game client, and will register an arbitrary score for the leaderboard.  You can defeat a trivial spoof attack by using some encryption and authentication on the transport, but it's difficult to make a really secure implementation..  and even if you do manage the secure implementation, it all hinges upon some sort of shared secret between the server and the game client -- a shared secret which a determined hacker could conceivably retrieve from the game client's executable (although you can certainly take steps to make that more difficult).  In any case, once he has the shared secret, he can write a spoofer to pretend to be the game.

Second, you have the "stomp", in which the attacker writes over the game client's memory, in order to convince the game that he actually has a much higher score than he earned.  You may have seen "trainer" patches for games which give you large amounts of cash or infinite health;  these work exactly the same way.  Write your own data into the spot where the game client stores your score, and you can make your score be whatever you want, and then the game client can perfectly legitimately upload that score to the game server.

How do you stop these?  Well, you can have the whole game be online, with the score never actually being stored by the game client (that way, the server always knows everybody's score, and doesn't need to trust the client to tell it the truth), that'd work and would be completely secure against these two most common types of attack.  Of course, it'd use a lot more server processor time and bandwidth.  Another (similar) approach would be to have the client game upload a replay of the game to the server, along with the high score;  if the game server can run the game replay and reaches the same final score that was submitted, then you know the score was legitimate.  Same problems with that approach too, though.

And even if you did one of those, there are other attacks you'd still be vulnerable to.

I don't have a better solution at the moment.  I imagine that if I ever make a game that'll use an online high score, I'll make the whole game server-based, to entirely avoid the problem.  I suspect that using an open source library would actually make the hackers' job easier.  While it's definitely true that one shouldn't rely on security through obscurity, a little bit of obscurity is still a good thing, as long as it's obscuring a well-engineered security scheme.  Smiley
Logged
bateleur
Level 10
*****



View Profile
« Reply #2 on: January 27, 2010, 01:45:29 AM »

You need to recognise that it's an arms race, so no strategy will be perfect. What I currently use involves the following tricks, plus one other which I don't wish to discuss because it's a security-through-obscurity measure which would be invalidated by publication.

* When a player starts a game, the client contacts the server for a scoring token. At the end of the game, the client submits two things: 1) The scoring token. 2) The score, scrambled using that token (encrypted would be even better, but I'm too lazy). The server then checks the scoring token for validity, then unscrambles the score and displays it if valid. The score token is then invalidated.

* The score number is held within a subset of the bits of a particular (uint) variable. The remaining bits form a checksum which is later checked by the server. A score not generated by the game software has only a very small probability of appearing valid.

* Highscores are only reported via the game client. Each game client has a cookie-like ID which it sends to the server with all comms. If a client sends an invalid scoring report it will be displayed to that client as though accepted. (This is more of a psychological than technical solution. There's sometimes no better way to make people give up than to let them think they've succeeded!)
Logged

Hajo
Level 5
*****

Dream Mechanic


View Profile
« Reply #3 on: January 27, 2010, 02:59:01 AM »

* When a player starts a game, the client contacts the server for a scoring token. At the end of the game, the client submits two things: 1) The scoring token. 2) The score, scrambled using that token (encrypted would be even better, but I'm too lazy). The server then checks the scoring token for validity, then unscrambles the score and displays it if valid. The score token is then invalidated.

I don't see how this helps? A cheater can make a fake client which gets a token, uses a reverse engineered scrambling algorithm, and then sends a seemingly "valid" score to the server. Admitted, it takes some skill for the scrambling part, but hackers seem to be good at it.

I'v been thinking about this problem since a while, but gave up because there didn't seem to be a safe solution. Terry once told me "don't worry, most people are too lazy to hack mediocre security", so maybe I'm just worrying too much. Terry's "Self Destruct" had a online leaderbord that time, which looked unhacked.

But maybe some good ideas pop up in this thread Smiley
Logged

Per aspera ad astra
mewse
Level 6
*



View Profile WWW
« Reply #4 on: January 27, 2010, 03:04:15 AM »

* Highscores are only reported via the game client. Each game client has a cookie-like ID which it sends to the server with all comms. If a client sends an invalid scoring report it will be displayed to that client as though accepted. (This is more of a psychological than technical solution. There's sometimes no better way to make people give up than to let them think they've succeeded!)

This is an awesome point.  The best way to make life difficult for attackers is to make it difficult for them to determine whether or not their attack has actually been successful.  Anything you can do to make them think that they've succeeded, when in fact they've been detected, will make your leaderboards less likely to be compromised.
Logged
alspal
Guest
« Reply #5 on: January 27, 2010, 03:07:15 AM »

You could have your game record the player as they play the game then it uploads the video to your computer so you can verify the score yourself.
Logged
brog
Level 7
**



View Profile WWW
« Reply #6 on: January 27, 2010, 04:28:17 AM »

You could have your game record the player as they play the game then it uploads the video to your computer so you can verify the score yourself.

This is more plausible than it may initially sound; if your game is deterministic (not hard to achieve if you record the RNG seed) you can easily record all input and check it with a 'light' version of the game (no rendering etc.) to see if a score is valid.  Perhaps more overhead than you want just for a high score table, but pretty reliable.
Logged
bateleur
Level 10
*****



View Profile
« Reply #7 on: January 27, 2010, 05:18:02 AM »

I don't see how this helps? A cheater can make a fake client which gets a token, uses a reverse engineered scrambling algorithm

Oh, absolutely! But that's why I describe it as an arms race. If the cheater is smart enough to reverse engineer your scrambling algorithm this will indeed fail, but that's a pretty big deal already. (They're more likely to just use your code directly by hacking your executable!)

To be honest, if someone smart enough to reverse engineer a scrambling algorithm wants to hack my game's hiscore table, I'd be quite flattered! Cheesy
Logged

Martin 2BAM
Level 10
*****


@iam2bam


View Profile WWW
« Reply #8 on: January 27, 2010, 05:23:55 AM »

Reverse engineering is a bitch. Specially with Flash decompilers.

Search the forums, there is a rad thread about this exact topic.

A mentioned approach that is awesome is don't letting the attacker know.
Defer removal of bogus scores to a couple of days later, but flag the IP as a hacker (after the N-th attempt, to account for legitimate scores gone wrong due to connection errors)
And then never believe that IP again (because he will eventually get it right and will not get noticed as a hacker)

Regards

Logged

Working on HeliBrawl
st33d
Guest
« Reply #9 on: January 27, 2010, 05:54:38 AM »

I already use a stomp stopper class:

Code:
public class HiddenInt {

private var _value:int;
private var r:int;
public function HiddenInt(start_value:int = 0){
r = (Math.random()*2000000)-1000000;
_value = r + start_value;
}
// Getter setters for value
public function set value(v:int):void{
r = (Math.random()*2000000)-1000000;
_value = r + v;
}
public function get value():int{
return _value-r;
}
}

The stomp method relies on using a utility like Cheat Engine to modify memory addresses.

You find those memory addresses by searching for a particular value in memory.

But if that value can only be extracted by subtracting a random number from it, that value can never be tracked. The value itself never exists in memory, it has to be combined with the random number to utilise it.

Since we started using this method in conjunction with encypted highscore submission, we stopped seeing ridiculous highscores on our games.
Logged
ThemsAllTook
Global Moderator
Level 10
******



View Profile WWW
« Reply #10 on: January 27, 2010, 06:02:19 AM »

Second, you have the "stomp", in which the attacker writes over the game client's memory, in order to convince the game that he actually has a much higher score than he earned.  You may have seen "trainer" patches for games which give you large amounts of cash or infinite health;  these work exactly the same way.  Write your own data into the spot where the game client stores your score, and you can make your score be whatever you want, and then the game client can perfectly legitimately upload that score to the game server.

I wrote a game with an online leaderboard once, and it was stomped during testing (fortunately by someone I know and could work with to fix the vulnerability). What I did was store certain sensitive numbers in memory (like the current score) as XORed with a random number that I store elsewhere, so except when I was actually reading or writing the value, it couldn't be found/modified trivially.
[EDIT] Beaten by st33d.  Smiley

However, alastair jack/brog's suggestion of recording the player's inputs and verifying them on the server sounds like a much better solution...
Logged

Hajo
Level 5
*****

Dream Mechanic


View Profile
« Reply #11 on: January 27, 2010, 06:46:00 AM »

However, alastair jack/brog's suggestion of recording the player's inputs and verifying them on the server sounds like a much better solution...

Eventually the cheaters will make bots that play the game for them Tongue

The stomp prevention methods are good though. I had been thinking of checksums, but this is definitely better.
Logged

Per aspera ad astra
raigan
Level 4
****


View Profile
« Reply #12 on: January 27, 2010, 07:21:16 AM »

If you have a replay system that's based on recording player input (so that "viewing" a replay is really re-simulating the whole game, with pre-recorded input being fed to the game in place of the keyboard's current state), then you can store replay data with each highscore.

You could then write a "spider" which scans over all non-verified highscores, loads+runs the replays, and compares them with the submitted score -- if they match, the score is verified, otherwise it's deleted.

This seems like it would be "perfect"/unhackable, but it relies on a deterministic simulation where replay data from any computer can be used on any other computer. Also, it's still somewhat hackable in that players *could* theoretically use external tools to make "Tool-Assisted Speedrun" replay data. At that point I'd be willing to give them a highscore though, they've earned it Smiley
Logged
Bennett
Jinky Jonky and the Spell of the Advergamez 3
Level 10
*



View Profile
« Reply #13 on: January 27, 2010, 07:46:24 AM »

I was planning to implement something along these lines, where player replays are downloaded as AI competitor 'ghosts', and checked in realtime while a living player plays. If the ghost's score doesn't match the stored score at the end of the round, then it is deleted from the high score table.

This way you don't need to have a spider or a light version of the code, you just offload the verification workload to the client computers.
Logged
brog
Level 7
**



View Profile WWW
« Reply #14 on: January 27, 2010, 09:12:39 AM »

Using players to check high scores is an excellent idea.

Another bonus could be that you could let players download the replays and see what tactics the top scorers are using.

HOWEVER the problem with both of these is that they'd give other players access to the replay data, allowing them (with a bit of hax) to submit them as their own scores.  This isn't insoluble though - you could just detect duplicates, or you could be a bit cannier and seed random numbers off the player's name, so that the same exact input wouldn't work unless you're going to submit the score as a player with the same exact name, which defeats the point.
Logged
Bennett
Jinky Jonky and the Spell of the Advergamez 3
Level 10
*



View Profile
« Reply #15 on: January 27, 2010, 09:57:28 AM »

Yeah I would seek to detect duplicates. The chance of a *top score* being a duplicate of another top-score would be very small.

But more importantly, copying another player's replay data wouldn't net you an insurmountable score, just the same score as another player.
Logged
bluej774
Level 1
*


It smells like wet fur in here!


View Profile
« Reply #16 on: January 27, 2010, 11:34:14 AM »

I was planning to implement something along these lines, where player replays are downloaded as AI competitor 'ghosts', and checked in realtime while a living player plays. If the ghost's score doesn't match the stored score at the end of the round, then it is deleted from the high score table.

This way you don't need to have a spider or a light version of the code, you just offload the verification workload to the client computers.

That is a brilliant idea.  However, I worry about the overhead of recording play input, especially for something like a long play in a shmup.  Strategy or simulation games wouldn't be difficult to verify.  Same with most RPGs.  But those types of games aren't really the type to commonly utilize high score tables.  Twitch games are what most commonly utilize high scores as the motivator for the player to continue playing.  And that could result in a huge amount of replay data.

I don't know that collecting replay data, as close to perfect as it is, could be implemented in a portable library, though.  Collecting replay data seems a little too invasive in the game development process to be achieved via outside library.

mewse is right that security through obscurity is a bad thing.  Punkbuster relies on that approach, but then again Punkbuster isn't fool-proof.  However, there are plenty of completely open source security software that works.  For example, Snort, OpenSSL and OpenSSH.
Logged
John Nesky
Level 10
*****


aka shaktool


View Profile WWW
« Reply #17 on: January 27, 2010, 11:41:17 AM »

I have implemented a replay feature like this, using input recordings, in a "twitch"-oriented game. And it worked beautifully and the recordings were very small in terms of file size. EXCEPT there were occasional floating point errors being handled differently on different platforms, throwing the simulation off completely, making the whole thing nearly worthless. Damn it.

If you ever do this, do not use any floating point calculations in game logic, or specifically target only one platform (i.e. don't use Flash or Java).
Logged
st33d
Guest
« Reply #18 on: January 27, 2010, 01:13:03 PM »

I've generally found Flash's Number to be fucking horrendous at even the most basic math. Fucking up simple things such as just removing 0.1 from a value that was an integer to start with.

Are you telling me it's a platform thing?
Logged
John Nesky
Level 10
*****


aka shaktool


View Profile WWW
« Reply #19 on: January 27, 2010, 01:27:02 PM »

Flash has a tendency to convert ints to Numbers without warning, but otherwise it uses the same double precision floating point math that any other software on your computer does and should not suffer any unexpected loss of precision. But that's not what I was talking about.

The hardware that performs floating point math, the FPU, implements a standard for how to handle floating point math. The problem is that the standard is not entirely specific regarding how the least significant digits should be handled, so not all manufacturers implement it identically. However, if your game only runs on an XBox 360, for example, you can rely on it always using the same FPU.

Also, frame rate independent engines are not compatible with input-based replays. Not without some serious forethought anyway.
Logged
Pages: [1] 2 3
Print
Jump to:  

Theme orange-lt created by panic