Here's a handful of thoughts about the state of human–music interaction.

Let's say you want to start making music. Maybe you're tempted to get a guitar. That's actually a surprisingly bad choice. It would take you several years of sustained effort before you're comfortable *making* music with your guitar. Until then, all your attention would be focused on the mechanics of *playing* guitar. It's a road paved with hating the guitar, hating your own fingers, hating whatever song you're trying to play. (youtu.be/9DbUPjEbIvA)

The guitar, like most traditional instruments, is a crappy user interface for music. The achievement of its design is that it's able to make good sounding tones at all, not that it's easy to play. As a guitar player you're forced to adapt your body after the physical requirements of that design.

This is why I'm so excited about computer based interfaces for making music. For the first time in history, we have the potential to design a music interface that works more like the way the human mind already understands music. After all, if you can listen to music, and understand it, you should also have the capacity to express music, given the right music making tools.

But the current crop of "serious" software in this genre is, surprisingly, almost not at all about making music. Instead, these apps are highly intricate *sound* making toolboxes, and it just so happens that you could choose to arrange those sounds into music, if you know how.

As a user, you need to have an extremely technical approach to your creativity, in two ways: you need to learn how to navigate these complex tools with their alien terminology and antiquated metaphors, *and* you need to supply the music theory and understanding needed to actually create music, rather than strange noise. This isn't much of an improvement over the guitar.

The problem is that these tools force you to work at the wrong level of abstraction. They supply an abundance of detail-oriented tools for adjusting aspects of the sound that no one (literally) can hear, but there are rarely any tools for working with the musical structures that really matter. In graphics software, we've always had tools for drawing circles, lines, curves, writing text, and inserting other meaningful symbols that are the basic building blocks of almost all visual work. Yet there's nothing like that for working with harmony, melody or rhythm in music apps.

Another big problem is that these tools are typically completely linear. This directly contradicts the way people make and play music outside of the computer: by jamming, experimenting and improvising, repeating things with variation, combining different ideas in unexpected ways. Mistakes and creative dead-ends are usually penalized by these user interfaces. It's as if you're expected to write and arrange the complete song first, then start building it out in the app starting with the intro and moving linearly to the end. Crazy.

This is the most interesting design problem I can think of. I think there's an incredible solution waiting to be discovered.

//-----------------------------------
If it were easy to make good music, 

//---
...more people could fit musical creativity into their lives, and the world would be a slightly better place?

--------------

What an apropos discussion. This week on designdare I wanted to write about something I saw last week. And then lo and behold someone started the discussion already. Neat!

Last week I saw a thing called called EarSketch:
http://earsketch.gatech.edu

The site itself is pretty academic, but it was shown by someone who worked on it, and it's really freaking cool. Here's the thing:

If you want to teach code to high school students, you're in for a challenge. If you want to teach code to high school students who aren't geeked about computers, the challenge is larger still. So the standard approach to teach code is to lead with games.

Games! Everyone loves games! It's a relevant cultural touchstone, the great equalizer!

Well, it turns out that's not the case for inner city kids in Atlanta. What this team discovered is that a much more culturally relevant angle is music. Lead with music to reach kids and get them interested in code. It's simple, but what an amazing insight!

And think about it - it takes a fair amount of fiddling to get code to power a game that has a satisfying end result. For example, if you start with a blank document and a canvas, the best you can do is something like:

walk(right);

... and then you'd compile that and watch a dude walk across the screen. Ok, that's all well and good, but building all the systems and constructs to make an actual, honest-to-goodness GAME can actually feel kind of intense.

Also, a computer-geek type person may see the dude walking across the screen and think "wow imagine what I can do with this little bit of knowledge!" ... but most people wouldn't. They'd see a guy walking across the stage and think "well that was sort of boring".

Now compare this to a line of code that says:

play(jazzLoop2);

So you hit play on that and what happens? You hear something. Ok, now maybe you add something else and set the tempo:

play(dubstepDrums8);
tempo(120);

Boom, you've got a song.

##But Wait, There's More

But remember, this class isn't about music. It's about learning to write code. So they've built this brilliant syllabus where they met with the compsci school and the music school and found a way to take musical concepts and use them as a way to address all sorts of code concepts. This is awesome.

And remember, every step of the way, you're able to hear actual music coming out of your actual speakers. They hooked up with Jay-Z's producer to make sure the loops are awesome, too. Nice touch.


So while I was watching this presentation, I was thinking about the traditional "drag and drop" approach of ACID, ProTools, Garage Band, etc. It encourages "painting" layers on top of each other. It leads to a certain kind of music making. But I agree, it's almost like the song should already be done before you start using the software.

But what about a command line? What about code? What about telling the code:

hey I want the tempo to be 130
and I'd like the song to last 3 minutes long
fade in the first 8 seconds and
fade out the last 30 seconds
So the main song is in C
But the chorus will change to A

... and so on. I feel like you could get the scaffolding of a song in a totally different way than we're used to with this linear stuff. You're sort of describing the vision for the song before diving into the specifics of it. I think there are enormous implications here.

And as was mentioned above, there's a huge opportunity for harmony, melody, and rhythm to be treated the way Photoshop treats elemental shapes in graphic design. With our without my code idea. It's all sort of tackling the problem from a different direction, which will lead to different creative results, which is awesome.

##It's More Than Just Music

This got me thinking of the good old command line. The pendulum has swung *way* too far towards buttons, in my opinion. Take something like Photoshop or Excel. Millions of features and properties and things you can do and blah blah blah and we decided to put everything behind a tiny little icon and hope for the best. Augh. Professional apps are such a pain.

I'm not saying you should have to learn scripting to learn Photoshop. I'm not saying everyone wants a command line everywhere. But I do think people consider their tasks in plain English, and all this button mashing makes you think like a robot, not like a person.

Example:

I want to be in Photoshop and be able to say "Export all documents as png". There are three things there. Export is the verb. All documents is the scope. As PNG are the options. Think about the complexity of that command today. We're making people think in terms of buttons, not in terms of task completion.

On my phone, I want to "Move all applications I haven't run in six months into a new folder", presumably to delete. That's like space age voodoo magic shit right there. But it's a real case! People would totally do that.

In iTunes, I want to "Play a mix of new music". Sure, you can get to that today by thinking like a damn robot. But there's this whole conversion between the very common human need "new music please" and actually making a smart playlist that's sorted based on year.

I want to say to the Finder "all files in this directory should rotate 90 degrees". It's possible if I know Automator or Applescript, but we're thinking like geeks instead of humans.

And I think it ties in with this music thing. We keep thinking like software engineers, not humans.

##Voice is Stupid

Voice control is bullshit. Always will be. It's an awful affordance. You don't know what's possible, it's constantly mis-understanding you, blah blah blah. I'm afraid we're going to have to wait for the baby boomers to die off before we get past this Star Trek vision of the future. It's myopic.

I don't want to string together a series of things just to have it probably fail. Instead, I want to see the commands I can do, in plain text. (not hidden behind a dumb icon). So imagine I'm in Photoshop and I just type "Rotate" into some sort of command line.

No reason for Siri to say "I didn't quite catch that" or "Do you want me to search the web?" or do the phone tree thing: "I heard you say rotate, now tell me what you want to rotate". No, it should be like auto-complete on Google.

So I type Rotate and it somehow tells me I've typed a valid verb. Great. Next I see "All documents", "this document", or any other options. I pick "all documents" and then it autocompletes the next bit: "90 clockwise", "90 counter-clockwise", etc.

This isn't about asking people to learn code or think like a robot. It's about unlocking pretty powerful (yet useful even for a computer novice) stuff behind plain English auto-completed prompts instead of making people hunt for icons that could possibly get them to their goal.

##It's Everywhere

This wouldn't just help music. It wouldn't just help software. It's how humans should use computers. Even Google is just dumbly returning pages that have the best SEO. No, I want to type "Best rated camera under $99" and know it's actually thinking on my behalf and returning a true answer.

Today it's just ads. Dumb little icons in software and ads online. One day we'll look back and realize how primitive it all is. Until then, we wait.

/////

Have you ever used Autocad? It works exactly as you've described. There's a commandline along with button tools. Each action you make with a tool is interpreted as a command and is shown below. You can also type commands to draw and transform, which is very useful -- because you're plotting a precise draft. 

This example reveals a harsh requirement for command-line interfaces. To use it, you must know a precise verbal or digital formula of what you need. So it works great when you need to rotate an image to 90 degrees. But what if you need to rotate it in a way to make that building stand straight? This is also a precise task, but it can't be expressed in numbers without a preliminary measurement. 

I think this is the case for most of the things humans do. A painter lays precise strokes on canvas to create the right image. A driver turns the wheel to a precise degree to make a good turn. We even have to make lots of precise movement to serve ourselves a cup of tea. And there is no verbal or digital interpretation for these things. 

Same things apply to music making. I believe music streams from feelings and impressions. So the verbal part of the brain just does not take part in the process very much. A composer can't describe his music, he can only write down what he hears. 

If music is a sequence of musical tones, then a piano keyboard is surely a perfect interface. It gives a composer direct and precise control over each tone, it's pitch, and length, and loudness -- all in a press of a key. 

Computers made things much simpler. Now you can record music as you play; correct a note if it was wrong; try different instruments, and overlay voices recording them one after another. These are fantastic tools composers couldn't even dream of some 100 years ago. 

But where are the great new composers who came to light thanks to these new tools? Where are lots of great artists and photographers, who gained a chance to show their talents with these fantastic digital tools?

I can't recall many. 

It seems that in art tool mastery is not the most difficult part. If you are an artist, you will learn to play many instruments if you need it; there's nothing too hard in that. If you can't make music, no tool will help you. 

--

Respectfully disagree. C helped people who couldn't write lower level code. Hypercard got people involved who never thought they wanted to be programmers. Flash was hugely influential in getting designers onto the web and working with technology. Or even hammers. Before hammers, people built stuff. Hammers just made it easier. Well-designed tools lower the barrier to entry. It's their job.

And who are the great new artists? They're all around you, and in 50 years we'll look back at this age like it was some special time compared to the crap in 50 years. There's nothing special about this time, just like there was nothing special about 50 years ago or in 50 years. All that will change is that people love to hate on the present, and over-idealize the past and future.

"We're getting soft!" is conventional wisdom. And it has been for literally thousands of years. But the real artists don't notice. They're busy making the masterpieces that we'll one day look back at as proof about how the best stuff has already been invented. And to get those masterpieces, to really break new ground that we haven't seen before, we'll make new tools.


--

@ "If music is a sequence of musical tones, then a piano keyboard is surely a perfect interface. It gives a composer direct and precise control over each tone, it's pitch, and length, and loudness -- all in a press of a key."

To continue the analogy with graphics software, the keyboard is like the 1px pencil tool. There are times when that is the right tool. But most of the time you're working with higher level concepts, like drawing a circle or typesetting text. It would be impossible to do a good job there if you had to manage the individual pixels.

The other day I bought Logic Pro X. The new MIDI effects are part of what caused me to write the OP. (I know such tools have been around in other DAWs before, but I'm a long time Logic user so it's the first time I can really try it as part of my workflow.) I found that even though the new arpeggiator is pretty crude, it gives me a whole new way of thinking of my arrangement. It allows me to work with harmony and rhythm as two separate things, which I find incredibly liberating. Tools like these are so important that they really belong in the piano roll itself. As a plugin, they become limited and feel like an afterthought.

It's strange how most DAWs treat their sequencer as a MIDI editor, rather than a music interface. MIDI is an implementation detail and I couldn't really care less about it. It's like if Illustrator was an SVG editor, just providing a 1:1 GUI for the features of SVG. Instead, Illustrator gives you tools for illustrating, and then allows you to export it as SVG if you need to.

The perfect music making scenario is really to have a band of competent musicians at your disposal. You could tell a good guitarist what to play by making the rhythm you have in mind with your mouth, or playing it crudely on your instrument of choice. If she gets it wrong you just give slightly refined instructions. Once she has the basic rhythm down, you can give her a chord progression, and she'll be able to change the chords without changing the phrasing, and without requiring new instructions on how to play. I can instruct her to add little details and flurishes and change things as we go along, and eventually I can tweak every single note she plays, if I want to.

Anyone could do this, regardless of musical experience. That's the kind of scenario I would like to see music software emulate. Give up initial control of the "1px" detail, to quickly get 80% of the way and capture the idea or feeling you have. Then you can go in and fix the precise details if you need to.

The current popular software really doesn't allow this at all. The closest thing I've seen (although I haven't tried it) is probably Synfire (see cognitone.com). Imagine the spirit of that but simplified and focused on popular rather than classical/art music.

(An interesting side track: There's a cultural problem here. People seem to appreciate music more if making it seems like a difficult and slightly magic thing. This is a very destructive sentiment. And unfortunately it might make some people turned off of the idea of giving the computer some musical intelligence.)

(@ the thread about the "great new artists", I can only agree with the previous writer. The majority of all music I have ever loved would probably not have been possible without computers.)