Allegro.cc - Online Community

Allegro.cc Forums » Programming Questions » Retrieving text from a Windows server as a Mac format file?

This thread is locked; no one can reply to it. rss feed Print
Retrieving text from a Windows server as a Mac format file?
AMCerasoli
Member #11,955
May 2010
avatar

Hi there... It's me again :-/

So... I'm trying to retrieve my "scores" file from the server, but I'm getting an extra "Carriage return" sequence.

This is the original file on my PC:

604140

An this is what I get when I retrieve it from the server.

{"name":"604141","src":"\/\/djungxnpq2nug.cloudfront.net\/image\/cache\/3\/4\/3499110885de8bed2e8a47563f8f4038.png","w":321,"h":327,"tn":"\/\/djungxnpq2nug.cloudfront.net\/image\/cache\/3\/4\/3499110885de8bed2e8a47563f8f4038"}604141

I'm saving it with a normal ofstream

Notepad says it's a Macintosh file... But I think there is something wrong when I manipulate the data...

This is what I'm doing:

#SelectExpand
1 void tabla_tiempos(ALLEGRO_USTR *array[15]){ 2 3 CURL *curl; 4 CURLcode res; 5 curl = curl_easy_init(); 6 7 std::string string; 8 9 if(curl){ 10 11 curl_easy_setopt(curl, CURLOPT_URL, "http://www.cerebrospain.com/script.php"); 12 curl_easy_setopt(curl, CURLOPT_POSTFIELDS, "name=lllñññ&project=curl"); 13 curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, write_data); ///Here I'm calling the Function write_data. 14 curl_easy_setopt(curl, CURLOPT_WRITEDATA, &string); 15 16 res = curl_easy_perform(curl); 17 curl_easy_cleanup(curl); 18 19 } 20 21 std::stringstream procesador(string); 22 23 for(int a=0;a<15;a++){ 24 25 char temporal[50]; 26 procesador.getline(temporal, 50); 27 28 array[a] = al_ustr_new(temporal); 29 //al_ustr_append_cstr(array[a], temporal); 30 31 } 32 33 } 34 35 static size_t write_data(void *buffer/*The data*/, size_t size, size_t nmemb, std::string *userp){ 36 37 size_t n = size * nmemb; 38 userp->append((char *)buffer, n); 39 40 std::ofstream outfile ("check_it.txt"); 41 outfile.write ((char *)buffer, 50); 42 outfile.close(); 43 44 return n; 45 46 }

And this is what the game shows:

604142

I don't know in which step I'm adding that extra CR.

And the PHP script simpler couldn't be:

<?php
$myFile = "scores";
$fh = fopen($myFile, 'r');
$theData = fread($fh, filesize($myFile));
fclose($fh);
echo $theData;
?>

Thank in advance...

Matthew Leverton
Supreme Loser
January 1999
avatar

And the PHP script simpler couldn't be:

It could:

<?php readfile('scores');

It's possible that the fopen without the 'b' flag is causing problems. But I'd switch to the above regardless of that.

I assume your script does other things too... Because if you are just returning a raw file, there's no need for PHP at all.

AMCerasoli
Member #11,955
May 2010
avatar

Damn!... That's a compression...

But it isn't working neither, the problem comes from my server, when I use ifstream to load a file from my PC that doesn't happens.
<Edited:>
So I have two options, the file comes modified, or <code>std::stringstream procesador(string);</code> modify it.
</Edited:>
I could do something like:

if(al_ustr_has_suffix_cstr(array[a], "\r"))
   al_ustr_remove_chr(array[a], al_ustr_offset(array[a], -1));

But isn't what I want. I mean, you first add the damn \r and then remove it? :-/

If I could see the code bite to bite when I do the std::stringstream procesador(string); :o

I assume your script does other things too... Because if you are just returning a raw file, there's no need for PHP at all.

It's ok that I'm a noob man... but no that much... ;D

bamccaig
Member #7,536
July 2006
avatar

The new lines would be converted by the file IO layer. For example, if you try to write "foo\n" in Windows it will be written as "foo\r\n". :) If you want for the data to not be interpreted specially then you always have to make sure to open the file in binary mode.

main.c#SelectExpand
1#include <assert.h> 2#include <stdio.h> 3#include <stdlib.h> 4#include <string.h> 5 6void print_chars(const char * const, const int); 7void read_string(const char * const, char * const, const int); 8void write_string(const char * const, const char * const); 9 10int main(int argc, char * argv[]) 11{ 12 const char * const fn = "example.txt"; 13 char output[5] = "\r\n"; 14 char input[5] = {0}; 15 16 print_chars(output, sizeof(output)); 17 write_string(fn, output); 18 read_string(fn, input, sizeof(input)); 19 print_chars(input, sizeof(input)); 20 21 return 0; 22} 23 24void print_chars(const char * const string, const int size) 25{ 26 int i; 27 28 printf("%d", strlen(string)); 29 30 for(i=0; i<size; i++) 31 printf(" %X", string[i]); 32 33 putchar('\n'); 34} 35 36void read_string(const char * const filename, char * const input, const int size) 37{ 38 assert(filename); 39 assert(input); 40 41 // "b" for binary mode.
42 FILE * f = fopen(filename, "rb");
43 44 if(f == 0) 45 { 46 perror(""); 47 exit(1); 48 } 49 50 int len = fread(input, sizeof(char), size-1, f); 51 52 if(len) 53 input[len] = 0; 54 55 if(len < 0) 56 { 57 fprintf(stderr, "LIES! Couldn't read bytes from file.\n"); 58 exit(1); 59 } 60} 61 62void write_string(const char * const filename, const char * const output) 63{ 64 assert(filename); 65 assert(output); 66 67 FILE * f = fopen(filename, "w"); 68 69 if(f == 0) 70 { 71 perror(""); 72 exit(1); 73 } 74 75 int len = strlen(output); 76 77 if(fwrite(output, sizeof(char), len, f) != len) 78 { 79 fprintf(stderr, "LIES! Couldn't write %d bytes.\n", len); 80 exit(1); 81 } 82 83 fclose(f); 84}

>gcc -Wall main.c
>a.exe
2 D A 0 0 0
3 D D A 0 0

>

OR...

Output: "\r\n"
Input: "\r\r\n"

If you attempt to process text in binary mode then you will need to manually account for newline differences though. :)

AMCerasoli
Member #11,955
May 2010
avatar

Are you sure? why doesn't happen with std::ifstream with files saved on my PC?.
I'm doing exactly the same.

I tried to use it (ifstream instead of stringstream) to extract the lines but I think isn't possible. I wanted to see if that was the problem. But isn't. As you can see in this part:

     std::ofstream outfile ("check_it.txt");
     outfile.write ((char *)buffer, 50);

I'm saving the data without even touching it... Obviously if I load that file (check_it.txt) I'll get the same as if were loading the file directly from the server, or at least that is what I think... For that reason I'm 90% sure that is a problem that comes from the server... :-/ But at the same time I'm not sure, since if I show it on the console std::cout<<string<<std::endl; I'm not able to see any problem, but the console isn't reliable anyway... AHHHHWWW!!! >:(

bamccaig
Member #7,536
July 2006
avatar

I posted the wrong version of that program. The only minor change that was missing was:

-    FILE f = fopen(fn, "r");
+    FILE f = fopen(fn, "rb");

That is just to show you the true contents of the file after writing in text mode. :) Edited, highlighted, and fixed up as originally intended.

The fstream library should work the same exact way:

main.cpp#SelectExpand
1#include <fstream> 2#include <iostream> 3 4void print_chars(const std::string &); 5void read_string(const std::string &, std::string &); 6void write_string(const std::string &, const std::string &); 7 8int main(int argc, char * argv[]) 9{ 10 const std::string fn("file.txt"); 11 std::string output = "\r\n"; 12 std::string input; 13 14 print_chars(output); 15 write_string(fn, output); 16 read_string(fn, input); 17 print_chars(input); 18 19 return 0; 20} 21 22void print_chars(const std::string & string) 23{ 24 std::cout << std::dec << string.length(); 25 26 std::cout << std::uppercase << std::hex; 27 28 for(int i=0, l=string.length(); i<l; i++) 29 std::cout << ' ' << (int)string[i]; 30 31 std::cout.unsetf(std::ios::uppercase); 32 33 std::cout << std::endl; 34} 35 36void read_string(const std::string & fn, std::string & output) 37{ 38 char buffer[5] = {0}; 39 40 std::ifstream stream(fn.c_str(), std::ios::binary); 41 42 stream.read(buffer, sizeof(buffer) - 1); 43 44 int len = stream.gcount(); 45 46 stream.close(); 47 48 buffer[len] = 0; 49 50 output = buffer; 51} 52 53void write_string(const std::string & fn, const std::string & input) 54{ 55 std::ofstream stream(fn.c_str()); 56 57 stream << input; 58 59 stream.close(); 60}

>g++ -Wall main.cpp
>a.exe
2 D A
3 D D A

>

You wouldn't normally realize it in code alone because just as the newline byte is transformed by default when it's written, the newline byte(s) are also transformed by default when read. I'm not sure if Mac-style newlines are supported or not (I wouldn't be surprised if they weren't).

AMCerasoli
Member #11,955
May 2010
avatar

Thank you bamccaig, I think I almost got it.

Heard this, this is pure science, so...

IF! I request the file from the server it seems that I retrieve it correctly, as now I'm saving my files as Mr. bamccaig said (using std::ofstream outfile ("check_it.txt", std::ios::binary);) I'm not able to see any problem.

BUT! (in forums there is always a but) if I do something like this:

#SelectExpand
1#include <fstream> 2 3int main(int argc, char *argv[]){ 4 5 char buffer[50]; 6 7 std::ifstream lol_entrada("lol.txt"); 8 9 lol_entrada.read(buffer, sizeof(buffer) - 1); 10 11 lol_entrada.close(); 12 13 14 15 std::cout<<buffer; 16 17 18 19 std::ofstream lol_salida ("lol.txt"); 20 21 lol_salida.write (buffer, sizeof(buffer) - 1); 22 23 lol_salida.close(); 24 25 return 0; 26}

I don't see any problem neither. I'm loading a file from my PC called ";D.txt" extract its content and then save it again, I'm using a buffer whit a predefine size because this is just a test.

NOW! back to topic, I thought that when I save a file as binary I wasn't going to be able to read it using a text editor, which isn't happening, since I can perfectly see my "check_it.txt" file using notepad, "check_it.txt" would be the file downloaded from the server.

WHY! is this happening? I have NO idea...

But despite all that...

Even when I'm able to save the file without problem now, I'm still getting the annoying little box with the interrogative sing inside.

4bd46a7e411bb8787bd04c8878074d71.png

So that let us one place to look at... well actually let me :( .

and it's THIS one:

         char temporal[50];

         procesador.getline(temporal, 50);

         array[a] = al_ustr_new(temporal);

One of these lines is adding an extra NULL character at the end of the string...

char temporal[50];

This isn't.

procesador.getline(temporal, 50);

Cplusplus dat com said:

Characters are extracted until either (n - 1) characters have been extracted or the delimiting character is found (which is delim if this parameter is specified, or '\n' otherwise).

So... Mr. bamccaig said: "The new lines would be converted by the file IO layer. For example, if you try to write "foo\n" in Windows it will be written as "foo\r\n""

It's possible that getline finds the "/n" delete it but the "/r" rest there, and that is what I'm seeing in my GUI?

because al_ustr_new() doesn't add anything at the end of the string right?.

Wow... This is game programming?

Where can I find more info about this? I'm surly getting everything wrong again. :(

Conclusion:

That was exactly the problem! NICE!

Since the Windows sequence is \r\n and getline() uses \n as delimiter
any line with a Windows linebreak will end with \r. Use this knowledge
to reduce the work required. In my case:

      for(int a=0;a<15;a++){

         char temporal[50];

         procesador.getline(temporal, 50);

         array[a] = al_ustr_new(temporal);

         if(al_ustr_has_suffix_cstr(array[a], "\r"))
            al_ustr_remove_chr(array[a], al_ustr_offset(array[a], -1)); //Delete the \r

      }

weapon_S
Member #7,859
October 2006
avatar

Wouldn't the smallest fix be: procesador.getline(temporal, 50, "\r"); then?
Of course you're method is more robust... *Sees char[50]* ehm, I guess that's okay in this case.

Edgar Reynaldo
Member #8,592
May 2007
avatar

Errgggghhh. :(

Read and write the file as binary, using \n as your separator. After that, write your own GetLine function that doesn't interpret \n as carriage return or carriage return linefeed or linefeed.

bool GetLine(istream& is , string& s , char delimiter = '\n') {
   s.clear();
   bool assigned = false;
   while (1) {
      int c = is.get();
      if (c == EOF) {return assigned ;}
      if (c == (int)delimiter) {return assigned;}
      assigned = true;
      s.push_back((char)c);
   }
   return assigned;
}

If the istream is binary to begin with, you might not even have to write your own GetLine function.

Go to: