al_open_directory is broken for me as well. But so easy to work around I never bothered to try and fix it.
al_load_font works for me, loading a .ttf from the .apk. So yes, will be good to debug that.
I never used dumb (and my Android builds do not include dumb), but al_load_audio_stream with an .ogg file does work for me, loading from the .apk. I can rather easily add dumb to my script creating the .aar file , all I need is a link to the source code of dumb, and instructions how to compile it for Android, and I'd make the script download and compile it.
I have no idea about the debug settings of those .so files, they are created by Allegro's CMakeLists.txt 
This is the complete commands I used to create the debug libraries:
cmake /home/allefant/allegro/git -DCMAKE_TOOLCHAIN_FILE=/home/allefant/allegro/git/cmake/Toolchain-android.cmake -DARM_TARGETS=armeabi-v7a -DCMAKE_BUILD_TYPE=Debug -DANDROID_TARGET=android-24 -DWANT_DEMO=off -DWANT_EXAMPLES=off -DWANT_TESTS=off -DWANT_DOCS=off -DPKG_CONFIG_EXECUTABLE=/usr/bin/pkg-config -DOGG_LIBRARY=/home/allefant/android/toolchain-armeabi-v7a/lib/libogg.a -DOGG_INCLUDE_DIR=/home/allefant/android/toolchain-armeabi-v7a/include -DVORBIS_LIBRARY=/home/allefant/android/toolchain-armeabi-v7a/lib/libvorbis.a -DVORBISFILE_LIBRARY=/home/allefant/android/toolchain-armeabi-v7a/lib/libvorbisfile.a -DVORBIS_INCLUDE_DIR=/home/allefant/android/toolchain-armeabi-v7a/include -DSUPPORT_VORBIS=true -DFREETYPE_LIBRARY=/home/allefant/android/toolchain-armeabi-v7a/lib/libfreetype.a -DFREETYPE_INCLUDE_DIRS=/home/allefant/android/toolchain-armeabi-v7a/include;/home/allefant/android/toolchain-armeabi-v7a/include/freetype2
Reading though that CMakeLists.txt there is no Android specific settings. But it might be cmake messing things up once again, adding special flags when it detects Android. Alternatively, maybe that's how a debugger works on all platforms? I don't think I ever used the debug library of something I did not compile myself (and therefore had the path to the sources). Can you see if you maybe can tell your debugger the location of the source code?
Otherwise another option is to compile Allegro yourself (using the above commands), then it should hardcode the correct paths to your local sources.
[Edit:] This  has a section called "Symbol Directories", maybe you can try that?