Announcement

Collapse
No announcement yet.

Upload error in 7.01.0009

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • #46
    Ok I have been trying to work around this buggy situation, so here you have already parts of the solution since it may take sc years to realize that the generated code is wrong.

    Suppose my application is called grid_filestorage. Then I have the files grid_filestorage_doc.php, grid_filestorage_js0.php, grid_filestorage_form0.php, grid_filestorage_nmutf8.php and some others that are less interesting.

    In grid_filestorage_doc.php you will find this code (or similar according to your situation).
    Code:
    <?php
       session_cache_limiter("");
       session_start();
    
       include_once 'grid_filestorage_nmutf8.php';
    
       if (!empty($_GET))
       {
           foreach ($_GET as $nmgp_var => $nmgp_val)
           {
                $$nmgp_var = NM_utf8_decode(NM_utf8_urldecode($nmgp_val));
                $$nmgp_var = str_replace('**Plus**', '+', $$nmgp_var);
           }
       }
       if ($nm_cod_doc == "documento_db")
       {
           $NM_dir_atual = getcwd();
    ...
    This should be:
    Code:
    <?php
       session_cache_limiter("");
       session_start();
    
       include_once 'grid_filestorage_nmutf8.php';
    
       if (!empty($_GET))
       {
           foreach ($_GET as $nmgp_var => $nmgp_val)
           {
                $$nmgp_var = NM_utf8_decode(NM_utf8_urldecode($nmgp_val));
                $$nmgp_var = str_replace('**Plus**', '+', $$nmgp_var);
                $$nmgp_var = str_replace('**Ecom**', '&', $$nmgp_var);
                $$nmgp_var = str_replace('**Jvel**', '#', $$nmgp_var);
           }
       }
       if ($nm_cod_doc == "documento_db")
       {
           $NM_dir_atual = getcwd();
    ...
    You can verify that this is needed since the grid_filestorage_js0.php has the function nm_mostra_doc(campo1, campo2, campo3)
    Code:
    function nm_mostra_doc(campo1, campo2, campo3)
    {
        while (campo2.lastIndexOf("&") != -1)
        {
           campo2 = campo2.replace("&" , "**Ecom**");
        }
        while (campo2.lastIndexOf("#") != -1)
        {
           campo2 = campo2.replace("#" , "**Jvel**");
        }
        while (campo2.lastIndexOf("+") != -1)
        {
           campo2 = campo2.replace("+" , "**Plus**");
        }
        NovaJanela = window.open ("grid_filestorage_doc.php?script_case_init=<?php echo NM_encode_input($this->Ini->sc_page); ?>&script_case_session=<?php echo session_id() ?>&nm_cod_doc=" + campo1 + "&nm_nome_doc=" + campo2 + "&nm_cod_apl=" + campo3, "ScriptCase", "resizable, scrollbars");
    }
    Here campo2 is the filename, so you can see that the nm_nome_doc parameter has the value of the filename which is passed on to grid_filestorage_doc.php

    But we are not there yet.

    There is another peice that is buggy in grid_filestorage_doc.php. If you want to view a file instead of downloading it, then under ie and some other browsers it
    is normal to send the mime info along. The generated code from scriptcase does not handle that properly. Apparently the guys from netmake did not test with
    chrome, ie8, opera, maxthon, firefox, safari, ipad safari and various other browsers. I dont have the time to test every browser so I only tested the big names on
    various operating systems.

    The corrected code is:
    Code:
       if (is_file($trab_doc))  
       { 
           header("Pragma: public", true);
           header("Content-type: application/force-download");
           $sProtectedFilename = str_replace(array(' ', "'", '!', ',', '-', '+'), array('__SC_SPACE__', '__SC_QUOTES__', '__SC_EXCLAMATION__', '__SC_COMMA__', '__SC_MINUS__', '__SC_PLUS__'), $nm_nome_doc);
           $sProtectedFilename = urlencode($sProtectedFilename);
           $sProtectedFilename = str_replace(array('__SC_SPACE__', '__SC_QUOTES__', '__SC_EXCLAMATION__', '__SC_COMMA__', '__SC_MINUS__', '__SC_PLUS__'), array(' ', "'", '!', ',', '-', '+'), $sProtectedFilename);
           if (isset($_SERVER['HTTP_USER_AGENT']) && false !== strpos(strtolower($_SERVER['HTTP_USER_AGENT']), 'chrome'))
           {
               header("Content-Disposition: attachment; filename=\"" . $sProtectedFilename . "\"");
           }
           elseif (isset($_SERVER['HTTP_USER_AGENT']) && false !== strpos(strtolower($_SERVER['HTTP_USER_AGENT']), 'applewebkit'))
           {
               header("Content-Disposition: attachment; filename=\"" . $nm_nome_doc . "\"");
           }
           elseif (function_exists('NM_utf8_urldecode') && $nm_nome_doc != NM_utf8_urldecode($nm_nome_doc))
           {
               header("Content-Disposition: attachment; filename=\"" . $nm_nome_doc . "\" filename*=UTF-8''" . $sProtectedFilename);
           }
           else
           {
               //header("Content-Disposition: attachment; filename=\"" . $sProtectedFilename . "\"");   [color=red]//<<<<wrong this one will will cause R&D.doc to be downloaded as R%26D.doc  whcih is not what we want. We want the real filename.
               header("Content-Disposition: attachment; filename=\"" . $nm_nome_doc . "\"");
           }
           readfile($trab_doc);
       } 
    ...

    I havent yet figured out why I can not upload a file named file+test.doc I know that it is because of the + but for windows that is no argument. Such a filename is completely legit. The conventions in windows are here: http://msdn.microsoft.com/en-us/libr...47(VS.85).aspx and http://en.wikipedia.org/wiki/Filename

    I have to do some more testing on other browsers to see if it works. This is not the end of it since there are more bugs in the generated code. But so far a few bugs are fixed. Lets wait and see what scriptcase does with these details on how to fix the code... They have my email...

    Be aware: any character>=char(128) is not working, filenames with + % still wont upload.
    # and & seem to work. So expect a few changes in this code..

    I hope this helps someone....
    Last edited by rr; 01-29-2014, 08:45 AM.

    Comment


    • #47
      I wonder why sajax is even used instead of $.ajax{ ... from jquery. I know the sajax library but with jquery ajax calls scriptcase should be easier. One library less since sajax can be simply replaced with the already available jquery and more using up to date libs with more options.

      Comment


      • #48
        There I go again. My trusty xdebug revealed another error in scriptcase. When I have a grid with a filename field for uploading a file and I choose the following file: gpn-2002+228.jpg then it ends up in my grid_filestorage.php as sc_87e430a8_gpn-2002 228.jpg this is wrong. The + turned into a space. I kow the sc_... is prepended before the string but the conversion from + to space is not correct.
        This is the reason why uploading files with a + in it doesnt work (under windows server).
        So my next request: FIX IT ASAP...
        A piece of my generated code so that its easier to find it back:
        Code:
                if ('ajax_grid_filestorage_submit_form' == $_POST['rs'])
                {
                    $filename_ = NM_utf8_urldecode($_POST['rsargs'][0]);
                    $lastupdate_ = NM_utf8_urldecode($_POST['rsargs'][1]);
                    $lastupdate__hora = NM_utf8_urldecode($_POST['rsargs'][2]);
                    $filename__ul_name = NM_utf8_urldecode($_POST['rsargs'][3]);    //<<<<<debug this one!!
                    $filename__ul_type = NM_utf8_urldecode($_POST['rsargs'][4]);
                    $filename__salva = NM_utf8_urldecode($_POST['rsargs'][5]);
                    $filename__limpa = NM_utf8_urldecode($_POST['rsargs'][6]);
                    $nmgp_refresh_row = NM_utf8_urldecode($_POST['rsargs'][7]);
                    $nm_form_submit = NM_utf8_urldecode($_POST['rsargs'][8]);
                    $nmgp_url_saida = NM_utf8_urldecode($_POST['rsargs'][9]);
                    $nmgp_opcao = NM_utf8_urldecode($_POST['rsargs'][10]);
                    $nmgp_ancora = NM_utf8_urldecode($_POST['rsargs'][11]);
                    $nmgp_num_form = NM_utf8_urldecode($_POST['rsargs'][12]);
                    $nmgp_parms = NM_utf8_urldecode($_POST['rsargs'][13]);
                    $script_case_init = NM_utf8_urldecode($_POST['rsargs'][14]);
                }
        A similar thing happens if I choose a file called: 3d_scene106.jpg <<<the e with dots is special and above chr(128)
        this reaches on the server in $_POST['rsargs'][3] as sc_13ad220b_3d_scene?106.jpg <<<< WRONG thus, it should be in UTF-8 as showing the same name as above.

        So the real reason why it goes wrong lies probably in the fact that the pages are in iso8859-15 and php is in utf-8 and the conversion fails.. At least I am closer in finding a solution/hack...

        Apparently if I click my button then this is sent: rs=ajax_grid_filestorage_submit_form&rst=&rsrnd=13 91097566848&rsargs[]=&rsargs[]=&rsargs[]=&rsargs[]=sc_13ad220b_3d_scene%EB106.jpg&rsargs[]=image/jpeg&rsargs[]=&rsargs[]=N&rsargs[]=1&rsargs[]=1&rsargs[]=&rsargs[]=incluir&rsargs[]=&rsargs[]=&rsargs[]=Sc_num_lin_alt%3F%23%3F1%3F@%3F&rsargs[]=496
        As is noticeable this %EB is a exactly the character in my iso-8859-15 codepage of my browser.
        So when it reaches the php code it is not properly converted to utf 8. See there I may have finally found the big bug.

        So when your database is in iso8859p15 (oracle) then your webpages apparently appear in iso-8859-15 upon which there is NO CONVERSION from the http post, so %EB is not a proper UTF-8 character (see http://en.wikipedia.org/wiki/UTF-8 it is the Hangul Control character). Because the filename is screwed the converter code makes another character and the whole stuff fails further on.

        I think I have given more then enough detail for scriptcase to fix this asap......!
        Last edited by rr; 01-30-2014, 01:12 PM. Reason: renews insights

        Comment


        • #49
          My pages are are codes as follows:
          <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
          "http://www.w3.org/TR/1999/REC-html401-19991224/loose.dtd">

          <html>
          <head>
          <title>grid_filestorage</title>
          <META http-equiv="Content-Type" content="text/html; charset=ISO-8859-15" />
          ....
          So it is in ISO-8859-15 which is the same encoding my database is in.

          If I use the standard blueimp jquery fileuploader that i also used in scriptcase and I set my webpages to the same Content-Type and DOCTYPE then I can
          upload everything without problems at all. Every file gets properly written in the exact filename so files like 3d_scene107.jpg and 3d_scene106.jpg and gpn-2002+228.jpg are properly
          uploaded. All without any conversion to be done and all with the proper filename on my filesystem.

          The project properties holds a locales tab. Apparently this is the one that determines the hmtl characterset being used in the generated html.
          I think the conversion from ISO-8859-15 on the web to ISO8859P15 in oracle is going wrong. In fact in my case there is no conversion needed at all...

          Comment


          • #50
            Ok here is the reason why you CAN NOT UPLOAD a file with a + sign in it using scriptcase.
            All filenames are sent using sajax to a specific function this way: x.open(sajax_request_type, uri, true);

            See the code:
            Code:
                                            x.open(sajax_request_type, uri, true);
                                            // window.open(uri);
            
                                            sajax_requests[sajax_requests.length] = x;
            
                                            if (sajax_request_type == "POST") {
                                                    x.setRequestHeader("Method", "POST " + uri + " HTTP/1.1");
                                                    x.setRequestHeader("Content-Type", "application/x-www-form-urlencoded");
                                            }
            
                                            x.onreadystatechange = function() {
                                                    if (x.readyState != 4)
                                                            return;
            
                                                    sajax_debug("received " + x.responseText);
            
                                                    var status;
                                                    var data;
                                                    var txt = x.responseText.replace(/^\s*|\s*$/g,"");
                                                    status = txt.charAt(0);
                                                    data = txt.substring(2);
            Here x is an XMLHttpRequest object,
            sajax_reuqest_type is 'POST'
            uri is your path e.g. '/ont/hrm/mutform/grid_filestorage/grid_filestorage.php'
            The same function sajax_do_call has a post_data which in this case holds:
            rs=ajax_grid_filestorage_submit_form&rst=&rsrnd=13 91183719043&rsargs[]=&rsargs[]=&rsargs[]=&rsargs[]=sc_4eb32c21_3d_scene+106.jpg&rsargs[]=image/jpeg&rsargs[]=&rsargs[]=N&rsargs[]=6&rsargs[]=1&rsargs[]=&rsargs[]=incluir&rsargs[]=&rsargs[]=&rsargs[]=Sc_num_lin_alt%3F%23%3F6%3F@%3F&rsargs[]=55

            Well it should be obvious now why a character + isnt working... Since this is post data it should be urlencoded but it isnt.
            Saying that it is application/x-www-form-urlencoded and not url encoding the data is basically wrong.
            So ppl form scriptcase please verify and solve this... I'll see if I can work around it (using url encoding).

            If the code there is this loop:
            Code:
                                   else if (sajax_request_type == "POST") {
                                            post_data = "rs=" + escape(func_name);
                                            post_data += "&rst=" + escape(sajax_target_id);
                                            post_data += "&rsrnd=" + new Date().getTime();
            
                                            for (i = 0; i < args.length-1; i++)
                                                    post_data = post_data + "&rsargs[]=" + escape(args[i]);
                                    }
            So this code uses the escape function.
            This is incorrect! escape('+') is still '+' so another encoding should be used.....
            Itis wrong because that is sent with a post and thus + gets converted to space.
            Last edited by rr; 01-31-2014, 01:19 PM.

            Comment


            • #51
              Well I seem to be able to get the proper filename in my php by doing a string replace:
              So this code:
              for (i = 0; i < args.length-1; i++)
              post_data = post_data + \"&rsargs[]=\" + escape(args[i]);

              becomes this code (do declare s tho)

              for (i = 0; i < args.length-1; i++){
              s=escape(args[i]);
              post_data = post_data + \"&rsargs[]=\" + s.replace(\"+\",\"%2B\");
              }
              But now it fails a bit further on. Sending a file with a char>=128 still fails... Possibly because the sajax POST does not have character encoding with it...
              As usual I see no reply from scriptcase.. So I hope I can find a way around it somehow..
              Last edited by rr; 02-03-2014, 10:26 AM.

              Comment


              • #52
                Found something else wrong.... In the grid_filestorage_nmutf8.php there is this piece of generated code (it handles encoding of special characters).
                Code:
                    function NM_utf8_urldecode($str)
                    {
                        if (is_array($str))
                        {
                            return $str;
                        }
                        $aRep = array(
                                      '&' => '&amp;',
                                      '<' => '&lt;',
                                      '>' => '&gt;',
                                      '"' => '&quot;',
                                      "'" => '&apos;',
                                      '+' => '&#44,
                                      '' => '&Aacute;',
                                      '' => '&aacute;',
                                      '' => '&Acirc;',
                                      '' => '&acirc;',
                This looks wrong to me..

                More correct would be
                Code:
                    function NM_utf8_urldecode($str)
                    {
                        if (is_array($str))
                        {
                            return $str;
                        }
                        $aRep = array(
                                      '&' => '&amp;',
                                      '<' => '&lt;',
                                      '>' => '&gt;',
                                      '"' => '&quot;',
                                      "'" => '&apos;',
                                      '+' => '&#43',
                                      '' => '&Aacute;',
                                      '' => '&aacute;',
                                      '' => '&Acirc;',
                                      '' => '&acirc;',
                Since when is the + character number 44???? That is the , character...

                Comment


                • #53
                  Found something else wrong.... In the grid_filestorage_nmutf8.php there is this piece of generated code (it handles encoding of special characters).
                  Code:
                      function NM_utf8_urldecode($str)
                      {
                          if (is_array($str))
                          {
                              return $str;
                          }
                          $aRep = array(
                                        '&' => '&amp;',
                                        '<' => '&lt;',
                                        '>' => '&gt;',
                                        '"' => '&quot;',
                                        "'" => '&apos;',
                                        '+' => '&#44,
                                        '' => '&Aacute;',
                                        '' => '&aacute;',
                                        '' => '&Acirc;',
                                        '' => '&acirc;',
                  This looks wrong to me..

                  More correct would be
                  Code:
                      function NM_utf8_urldecode($str)
                      {
                          if (is_array($str))
                          {
                              return $str;
                          }
                          $aRep = array(
                                        '&' => '&amp;',
                                        '<' => '&lt;',
                                        '>' => '&gt;',
                                        '"' => '&quot;',
                                        "'" => '&apos;',
                                        '+' => '&#43',
                                        '' => '&Aacute;',
                                        '' => '&aacute;',
                                        '' => '&Acirc;',
                                        '' => '&acirc;',
                  The code after that screwes up the + character:
                  Code:
                          $str = str_replace(array_values($aRep), array_keys($aRep), $str);
                          if (isset($_SESSION['scriptcase']['charset']) && 'UTF-8' == $_SESSION['scriptcase']['charset'])
                          {
                              $str = mb_convert_encoding($str, 'UTF-8');
                          }
                          $str = preg_replace("/%u([0-9a-f]{3,4})/i", "&#x\\1;", urldecode($str));
                          if (isset($_SESSION['scriptcase']['charset']) && 'BIG-5' == $_SESSION['scriptcase']['charset'])
                          {
                              $str = @html_entity_decode($str, null, 'BIG5');
                          }
                          else
                  So now there I found the NEXT BUG... the urldecode should be rawurldecode !!! See http://nl3.php.net/manual/en/function.rawurldecode.php raw url decode does not decode the + sign.... And suddenly I can store filenames with + signs in it.

                  So what is left now is the proper en/decoding of any char>=128.
                  Last edited by rr; 02-04-2014, 05:47 AM.

                  Comment


                  • #54
                    Hello rr,

                    Issue reported to our bugs team.

                    regards,
                    Bernhard Bernsmann

                    Comment


                    • #55
                      aducom / rr

                      Did you guys ever get your own uploader working? If so, is it anything you can share?

                      Comment


                      • #56
                        I did experiment in detail with the jquery file uploader and did get it working http://blueimp.github.io/jQuery-File-Upload/
                        but it needs some tweaking also (file names go wrong on windows so it needs a patch which is on that website as well somewhere).
                        But it didnt really fit what I wanted, since I wanted to store extra data with every upload. Basically scriptcase uses this one also as basis, just an older version
                        with some unfixed bugs (talking about sc6, 7, 7.1 here now).
                        utf-8 filenames only work on utf-8 filesystem here or the filename needs conversion which is shown somewhere on that website.


                        quote from https://github.com/sstur/jquery-file...ked-Questions:
                        Is there a problem uploading files with non-ASCII characters (PHP, Windows server)?

                        If your non-ASCII file names get uploaded with strange characters like ä, ö or ü you probably need to apply the utf8-decode() method on the file names of uploaded files, e.g. by overriding the trim_file_name method:

                        <?php
                        require('upload.class.php');

                        class CustomUploadHandler extends UploadHandler {
                        protected trim_file_name($name, $type, $index) {
                        $name = utf8_decode($name);
                        return parent::trim_file_name($name, $type, $index);
                        }
                        }

                        $upload_handler = new CustomUploadHandler();


                        If you check your scriptcase installation you'll notice that the jquery plugin is being used.

                        Comment


                        • #57
                          Okay thanks rr - I'm having a little mess about. I don't need anything too complicated - but I do want it to work and be stable and consistent. Will post here what I end up doing (if I have the time to finish).

                          Comment

                          Working...
                          X