The game I’m currently working on, Motion Picture, is like a jigsaw puzzle where the pieces play bits of live video, rotated and inverted.
The main mechanics of the game I needed to implement, which I’m simplifying here (a lot of these have a number of boring sub-tasks), were as follows:
1. Divide a video clip into 16 pieces.
2. Allow the pieces to be rotated and inverted, both by the player and the game’s AI.
3. Allow pieces to be picked up, dragged, and then snap into a spot on the game board.
4. The game acknowledges when all the pieces are in the correct spots and rotations on the board.
Let me tell you how I did with each of these.
Divide a video clip into 16 pieces.
When I started development, I thought this was going to be the most challenging part by far. It’s an unusual thing to want to do. Most games that use video clips would play them whole, not break them apart. But it turns out that Unity allows you to tell an object to display a piece of a video on an object, as long as you get the math right on dividing up the video into sections. As a graduate of Emerson College, “getting math right” is not my strong suit, but with enough trial and error, me and my trusty Calculator app got the pieces to look good.
Allow the pieces to be rotated and inverted, both by the player and the game’s AI.
I figured out a version of this very quickly. My initial solution, which was to map the H (horizontal) and V (vertical) buttons to change the piece’s rotation property, worked immediately. Yay! But it was clunky. Boo! Sometimes the piece would accidentally rotate in 3D space, instead of 2D space, even though I setup the project in 2D. The piece would look like a character in Flatland. I found a much better solution digging through the Unity forums, which was to change the object’s “scale” property instead of “rotation.” Since I didn’t actually need to rotate the piece itself as an object, just the image “texture” on the piece, changing the object’s scale from “1” to “-1” or vice versa became a reliable solution.
The game acknowledges when all the pieces are in the correct spots and rotations on the board.
I didn’t think this was going to be hard, and by the standards of this particular project, it wasn’t. I had an image in my head of what the system would look like from the beginning. Every time a piece rotated, I had the piece tell a PieceManager object, “This is my current rotation and the space I’m on!” The PieceManager puts that information into two arrays (a kind of database) and then checks them. If all the rotations and spaces are correct in the arrays, then the game enters the “You Win” sequence. This was a job for variables, and I have a decent understanding of variables from working in Ink for my Adventure Snack games. Variables are data trackers, like Player Name or High Score or How Many Times Have They Pet the Cat So The Game Knows Whether To Unlock The “Purr-fect Player” Achievement.
Allow pieces to be picked up, dragged, and then snap into a spot on the game board.
I saved the worst for last!
My first experience with clicking and dragging was on the Apple IIgs computer we had growing up. There was a tutorial program called Your Tour of the Apple IIgs with an adorable pixel art jungle storybook vibe. I remember there being a number of mini-games, including one where you clicked and dragged an archeologist through tunnels, up a rope hovering over dangerous waters, and then over to his robot companion. Mouse controls came naturally to me, in a way that command line interfaces never did. Commands were a foreign language, which I had trouble learning. The mouse was visual, fun, and simple.
Maybe that’s why I took click and drag mechanics for granted when developing Motion Picture. The implementation of grabbing and moving an object using Unity and Playmaker wasn’t so hard. There are good tutorials for making it happen. Granted, it’s not so simple that I understood why the YouTubers’ solutions worked, but the tutorials made it easy to follow along and implement the logic. My issues – my downfall – was really about how to control what happens after the player lets go of the mouse.
There was a moment, about a month or so into the project, where I thought I was pretty much done. I got the pieces to show video clip sections, which I could move into their correct spots, and win the game! And then I noticed something. I accidentally let go of the mouse button too early while dragging. My piece landed on top of another piece. When I experimented, it turned out that I could stack all the pieces on the same space on the game board. This was not at all what I wanted at all! When I dropped a piece on an occupied space, I wanted the piece to go back to where it came from. Instead, any space had the potential for a confusing pile-up.
In Unity, there is this concept of a “collision,” which means two game objects are touching each other. (“Mom, tell GameObject to stop colliding with me!”) So if you can get Unity to successfully detect a collision, you can have it do something. For example, if two asteroid objects collide, then destroy both asteroid objects and play the explosion animation. But none of my collision detectors worked properly. I set up variables like alarms to say to me, “A collision was detected!” in the backend Console, but the alarms would not ring.
To solve this problem, I needed help. Luckily, I knew who I could ask. Gwen, my programming partner on Cocoanut Hotel, was a Unity ace before switching over to Godot. She agreed to help me solve the stacking bug with One Not-So-Weird Trick, but in doing so, observantly pointed out a much more insidious problem.
🎲 Your Turn: What are the kinds of challenges you run into making your art? Do you ever have to deal with unexpected challenges? Have you ever encountered a memorable video game bug? I want to hear from you! Reply to this email or hit the orange button below to leave a comment and share your thoughts with the whole world.
5 responses to “Clicked and Dragged”
I’ve never programmed anything beyond Basic (unless you count editing PHP and ASP, which I don’t really…) But I had a problem that I never could solve…
Back then, Electronic Detective (https://www.youtube.com/watch?v=9fWqmA9YjYc) was popular with a certain crowd (like, 12-year-old boys…) and I thought it would be fairly easy to port it to the C64 using Basic. And it was — I even had a flashy title screen to go along with it.
Thing was, it was always the same one or two suspects with the same weapon. I was, of course, generating all those variables using a random number generator. Turns out, it doesn’t work so well in Commodore Basic! I banged my head against a wall for months, but never solved it.
I heard later about generating a random number by checking the time to seven decimals or something. But by that time, the C64 was in a box…
Not sure if you’d call that a “bug” or just a limitation of the system. But here I am talking about it 45 years later, so it was definitely memorable!
~Graham
I messed around a lot in Basic, inputting code from magazines, but I never made my own working game in it. (I managed to do that in HyperCard and upload it to America Online.)
If I had gotten as far as you, with the title screen and everything, and was felled by Commodore Basic’s incompetent random number generation, I’d remember it, too!
Possible trigger warning: Mentions of concepts from highschool Algebra, Geometry, and/or Trigonometry classes. If you found HS math traumatic, this comment might not be for you… I was two years ahead of my grade mates in Highschool in maths and Precalculus was the first time I got below 90% in a math class, the subject matter of this comment comes from the evolution of me graphing stuff on a TI-83 for fun back in HS… Note, understanding the math isn’t needed to appreciate the end result, but the explanation is hard without mentioning the maths.
I dabble with programmatically generated art, final renders of which can be found at:
https://sightless-sanctuary.net/Graphics/Programmatic/
Mostly, I use the ASCII version of portable pixel map(.ppm) which stores pixel data as a whitespace delimited list of decimal RGB triplets, use a nested for loop in C++ to loopthrough some range of the XY-plane while treating each frame as a z-slice, and use some mathematical relationship to decide what the red, green, and blue values should be for the pixel with those coordinates. I then use optipng to convert the .ppm files written by my code to optimized pngs and then apngasm to assemble the frames into an animated png.
Some of the pieces I make, I have no clue what the result should look like and have to figure out what’s going on from descriptions sighted people give me when I share the results. Others, I have a very good idea of what the result should be, only for sighted commentary to reveal something went wrong and then have to figure out how to correct it, and since I’m literally blind, I can’t just tweak the code, compile, run, and then look at the ppms in an image viewer. Add in lack of reliable sighted assistance and each trial has to go through the lengthy process of converting all the images to optimized pngs(because ASCII ppms are too big to reasonably share via a personal website whose hosting budget is pocket change(literally, I’ve had individual pieces that produced 8GB of .ppm files, the entire offline backup of The Sightless Sanctuary is less than 250MB. I don’t want to know what a 2 orders of magnitude increase in my storage usage would do to my hosting costs), upload, and wait for sighted feedback.
In particular, the Radioactive symbol I made as an experiment in bounding regions by the value of theta from polar coordinates took several tries to get right, the first version missing the outer parts of the symbol entirely, the second attempt having the outer parts narrow and clustered, and only the third attempt getting things right… and if memory serves, some of the mistakes are still in the radioactive folder at the above link… and speaking of theta, some of my other early experiments with using polar coordinates in my programmatic art included the built-in C++ function I use to calculate theta from x and y outputting in the range -pi to pi instead of the rang 0 to 2pi, and it taking a bit to figure out the right way toad an increment to theta based on the z coordinate to allow for spinning animations, resulting in hundreds of identical frames.
I’ve certainly ran into other issues with the programmatic art, but those are examples from the most recent sprint of activity on this front… also, even though I’m using the same loop structure for just about every piece along with the same code to create and write to files, I haven’t been able to figure out a way to integrate all the different code snippets into one program instead of a bunch of bespoke programs with very high code overlap, and the best I’ve done for code that can produce variable output based on runtime interactions, the one thing where I’ve shared the code and not just the final render, is limited to picking a baseline value for each color channel plus coefficients to multiply the point’s x and y values by… I would love to build Color Graph up into something where I could just type in a wide range of functions in terms of cartesian and polar coordinates for each color channel and share binaries for all major platforms, but I’m kind of at a loss for interfaces more sophisticated then having the user type something that’s then treated as a text string and used as is or as a number I can then do math with…
Also, with the bead mat I showed off in a previous comment section and other beaded items I’ve made, I’m constantly dropping beads on the floor, only to find them later and have no idea which color I just picked up… and again, lack of reliable sighted assistance means I sometimes end up with a sizeable collection of mixed beads in need of sorting, and while my portable media player has a color reader app, I am not convinced its results are reliable, especially since it relies on a phone camera instead of it having a probe you can press to a surface that then works like the scan head of a flatbed scanner like some of the dedicated cloor reading devices I’ve tested in a vocational rehab setting… at the moment, the medium sized spice jar(about 2″ diameter) I’m using for reclaimed beads has maybe an inch deep pile of beads inside it… and the pony beads I’m using are advertised as 6*9 mm… considering these jars are the perfect size to hold a 500-count bag of beads, I’d say that’s at least 100 beads in need of sorting… Also, while making a beaded scarf, I mixed up my bead jars at one point and had to undo about half of a 108-bead patch to correct the pattern I was going for.
Most of what you wrote went over my head, because I’m not very math-y. (My HS allowed me to take Statistics instead of Trigonometry, which I actually enjoyed because of its practicality.) That said, I went through the archives of your art and it’s awesome! I particularly like ColorGraph/example.png and Absolute/absolute.png. The colors are so vibrant and mesmerizing.
Funnily enough, Statistics is probably the branch of mathematics I have the most trouble with, at least among the stuff required for a Bachelor’s in Computer Science… Took AP Statistics my senior year of highschool because Calculus couldn’t fit into my schedule and it was the first math class I got a D in… didn’t even bother sitting for the AP Exam, I did so poorly in the class, and it would be among the maths I had to take multiple times to get my college credit for it… Calculus 2 and its integrals also gave me trouble as did Linear Algebra and its matrix factoring, but I had no issue with the rest of Calculus or Linear Algebra, but struggled immensely with statistics.
Glad you like some of what I’ve produced. I know the maths behind the work isn’t for everyone, but part of what makes geometry one of my favorite branches of math is how much beauty seems to drop out of relatively simple equations if you aply the right translation of the abstract to a form our senses can make sense of… still, my metaphorical kingdom for an accessible graphing calculator and a way to make the output tactile. I work with color because its what the file formats I know how to write give me to work with, but it would be amazing to have a geometric design of mine carved into a plastic, wood, or metal plate or to hold a 3-d printed bauble.