Shouldn't al_lock_mutex() allow locking separately for reading and writing?

ala

al_lock_mutex(AL_LOCK_WRITE);
al_lock_mutex(AL_LOCK_READ);
al_lock_mutex(AL_LOCK_BOTH);

http://en.wikipedia.org/wiki/Readers%E2%80%93writer_lock

Not quite, it wouldn't be a mutex then. A mutex is to exclude other threads from access to a certain block of code, no matter if the block reads or writes.
What do you need a readers writers lock for specifically?

No, it's just a normal mutex not a read-write lock. You can easily implement a read-write lock using a mutex though. I think a read/write lock would be useful enough to add to Allegro's API, if someone does want to add it. However if you use C++ just use for example std::shared_lock/std::unique_lock.