I was just running an animation using frames. But when I changed the rotation the edged became distorted but for rotation = 0 degrees it's crystal clear and fine. I am using #define PI 3.141592653589793f for converting the degree to radian. The data type used to store the value in radians is a float.

What would be the potential cause of this problem?
All the graphical results are attached to this post. For visual analysis.

float Position::getRadRotX() const
{
return((rotX * PI) / 180);
}

Something you can do is turn on multi-sampling.

You have to enable it before creating your display :

al_set_new_display_option(ALLEGRO_SAMPLE_BUFFERS , 1 , ALLEGRO_SUGGEST);
al_set_new_display_option(ALLEGRO_SAMPLES , 4 , ALLEGRO_SUGGEST);

ALLEGRO_DISPLAY* d = al_create_display(...);
if (al_get_display_option(d , ALLEGRO_SAMPLE_BUFFERS) != 1) {
   cout << "Failed to enable multisampling on the display." << std::endl;
}
cout << "There are " << al_get_display_option(d , ALLEGRO_SAMPLES) << " sample buffers in use." << std::endl;

Thank you, Edgar Reynaldo for the replay

std::cout<< says:-
Failed to enable multisampling on the display.
There are 0 sample buffers in use.
And it's the same as before, blurred edges.

That's curious. What platform are you on?

Windows 10 64bit. Visual Studio 2017
Intel(R) Pentium(R) CPU N3710 @ 1.60GHz 1.60 GHz
Intel HD Graphics 405
Shader version 5.0
Open GL version 4.4
Open CL version 1.2
Direct X runtime version 12.0
hardware supported version 11.1

Okay, and what Allegro driver are you using? OpenGL or DX?

Show your display set up code.

void debug()
{
ALLEGRO_DISPLAY *display;
ALLEGRO_DISPLAY_MODE dispData;

al_init();
al_init_primitives_addon();
al_init_image_addon();
al_get_display_mode(al_get_num_display_modes() - 1, &dispData);

al_set_new_bitmap_flags(ALLEGRO_VIDEO_BITMAP);
al_set_new_display_option(ALLEGRO_SAMPLE_BUFFERS, 1, ALLEGRO_SUGGEST);
al_set_new_display_option(ALLEGRO_SAMPLES, 4, ALLEGRO_SUGGEST);

display = al_create_display(dispData.width, dispData.height);
if (al_get_display_option(display, ALLEGRO_SAMPLE_BUFFERS) != 1) {
std::cout << "Failed to enable multisampling on the display." << std::endl;
}
std::cout << "There are " << al_get_display_option(display, ALLEGRO_SAMPLES) << " sample buffers in use." << std::endl;

}

That's really weird, because I'm running mostly the same setup as you, and I know I have a working multisampling example around here somewhere. Let me find it.

Run a debugging build, and see what allegro.log has to say about the display.

EDIT
Try the OpenGL driver and see if it enables multi sampling. Could be a DX issue.

Thank you Edgar Reynaldo for helping out,
The log file is attached above.
Allegro produce log's in debugging mode only or it's the same in the release mode too?. I never noticed it before

See my edit. From allegro.log it looks like none of the display formats tested support multi-sampling. They all say "samples 0/0" in the log.

Try the OpenGL driver.

Don't know how to switch to open GL in windows 10.

al_set_new_display_flags(ALLEGRO_OPENGL | ALLEGRO_FULLSCREEN);

Thank you, Edgar Reynaldo for your kind support.
It's little better, and the sampling worked ....
But not fluid as the 0-degree pic.

This is probably a bug in the DX driver.

You might try to increase the samples to 8 but I think 2 or 4 will give the most optimal results.

How many samples did it end up with? What does allegro.log say?

I replaced the value of PI with const float PI = 104348 / 33215;
Now it seems better. Still some little distortions here and there.

I switched back to d3d and used the same code as "const float PI = 104348 / 33215;" and the result was the same as when the OpenGL was rendering with 4 buffers.

Are you drawing your sprite on the back buffer or another bitmap? If you're drawing it on another bitmap it won't be multi-sampled unless you enable multi-sampling on the bitmap as well.

al_set_new_bitmap_samples(8);
al_create_bitmap(w,h);

As for your float problem. int/int always gives you int. You need to specify a floating point literal value as such by using .0f on the end of it. Otherwise you get integer division, which discards the remainder.

Here's multi-sampling enabled through al_set_new_bitmap_samples :

https://www.allegro.cc/files/attachment/611523

Actually, 8 samples looks the best.

Thank you Edgar Reynaldo, for bearing with me ...
I am drawing to the back buffer (not to any bitmap). I removed the entire code for sampling and I got the same result as before with the sampling enabled. It's little weird

Okay, I'm thinking that MultiSampling doesn't do anything for sprite drawing. Try this instead : Before you load your sprites, call the following code :

al_set_new_bitmap_flags(ALLEGRO_MIN_LINEAR | ALLEGRO_MIPMAP);

ALLEGRO_MIPMAP won't do anything if the width and height of the image are not a power of two.

You can also try ALLEGRO_MAG_LINEAR, but it can make sprites blurry when enlarging them.

Thank you Edgar Reynaldo,
I think it's little better than before - just little distortion especially at the tip of the gun. For a starting, it won't be a problem. Thank you for helping out.

Well, I'm glad it's improved somewhat. Could you show a screenshot of the sprite with ALLEGRO_MIN_LINEAR enabled? Is it really any smoother?

I've never had this problem before, but I haven't been rotating sprites much. I tried out rotation on multi-sampled primitives and enabled down scaling and ALLEGRO_MIN_LINEAR.

While it doesn't do much for full size sprites, it looks pretty good when actually scaling down.

You might try increasing the resolution of your graphics and scaling them down in your game, but whether that is worth it or not is up to you.

I will agree, there are some rather jagged edges when rotating sprites, even multi-sampled primitive sprites.

Thank you Edgar Reynaldo,
My previous post had the pic's attached to it with these flags enabled " al_set_new_bitmap_flags(ALLEGRO_MIN_LINEAR | ALLEGRO_MIPMAP); ". But however the width and height of the png's are not a power of two, and the pic shows an unscaled version of the character. I will try to use HD images - and downscale it as per need if required.

One more thing - Should the actual bitmap have to be in the power of two or just the sub-bitmaps have to be in power of two.

If you want mipmaps, the source image has to be power of two in width and height (not necessarily the same though). mipmaps are really only good if you change the scale of your sprite often at runtime. That's what mipmaps are, pre-scaled versions of your image.

I will play with this some more over the next few days. Could you attach the image you're using for your player? I'd like to try some things.

Thank you Edgar Reynaldo,
So the source image has to be a power of two that's bad.......
I took all the game assets from open game arts https://opengameart.org/content/animated-top-down-survivor-player
To test the code I uploaded - I will attach the combined animation sprite with the custom script I am using to this post.

Note for the future - if you're going to attach multiple files, like a game project, it's best to put it in a .zip file so there's only one file to download. I will take a look at the assets.

You only need power of two textures if you want mipmaps. mipmaps are only good if you're planning to zoom in and out on your images.

EDIT
I made a small demo of your survivor guy being rotated and scaled with different combinations of ALLEGRO_MIN_LINEAR, ALLEGRO_MAG_LINEAR, and ALLEGRO_MIPMAP. Mipmapping didn't seem to help at all, and resulted in a faded image when scaling down. Mipmapping doesn't do anything when scaling up. Shrinking with ALLEGRO_MIN_LINEAR and growing with ALLEGRO_MAG_LINEAR really helped though, when not drawing at actual size.

Here's the demo - MinMag.zip.

For some reason, mipmapping made the images fade when drawn at less than actual size.

https://www.allegro.cc/files/attachment/611584

Rotate around the center with the mouse. Mouse wheel changes zoom.

Thank you, Edgar Reynaldo, for your kind support.
I will look into the code.
Yes, next time I will take special care in zipping all the contents before attaching it.
Thank you once again....

Note that your sprite has antialiasing. I'm not speaking of the borders (alpha on the fringe), but on the surface of the image, some pixel colors are drawn a spcific color in order to smooth the difference between pixels around it : for example a pixel can be set grey because it's between a black shape and a white shape. If such image is rotated at a similar zoom level to the original (*), the grey pixel may be preserved, and end up in a place which is not between a black and a white pixel, so it looks out of place.
This is a very simplified explanation (the rotozoom doesn't just juggle pixels), but the effect is real and can explain some oddities that appear at several angles but always at same position relative to sprite (ie. "right elbow", "corner of backpack"...)

(*) Scaling UP the sprite amplifies the problem, scaling it down makes it disappear very quickly.

Thank you, Audric for the replay
I never noticed that changes ...... Now I know; I will try to use good quality High definition images and will scale it down when I need to draw - Hope that fix the problem for now. My project is live and if you like to take a peek into the stuff i am doing or need to edit or suggest changes feel free to do so.
https://github.com/ThinkalVB/NearEarth
Thank you once again for the help...

For my Deluxe Pacman 2 game, I rotate the Pacman character, as the main image just faces the right. For my setup, all I really have is:

al_set_new_bitmap_flags(ALLEGRO_MAG_LINEAR);

I have a bunch of code commented out that I tried in the past, but this seems to be the one I ended up using that works well using DirectX and OpenGL.

Not sure if that would help or not, been a while since I worked on this sort of problem. And in my game the rotations are increments of 90 degrees.

Thank you Neil Roy,
I am trying a combination of MIP_MAP and MIN_LINEAR , I am more into zooming out, not much zooming in, but I have backed up all of your suggestions for future references if a need arise. Thank you once again for leaving your valuable comments in here.

Not relevant, but something you should know: the only minification filter you should ever use is some sort of mip mapping. Anything else will cause you to thrash the texture cache.

Thank you relpatseht,
Almost forgot that fact.