Posts Tagged ‘workaround’

One of the optional input for the creation of the classification models in OrfeoToolbox is XML image statistic file that is produced by Compute Image Second Order Statistics tool. If you opt to calculate this statistics – necessarily check the created XML file. If you will see “#INF” record in band statistics (i.e. “1.#INF”) – replace “#INF” with something like “0e+32” (“1.#INF” -> “1.0e+32”), or (a better solution) – calculate the statistics for this problematic band independently (probably with another tool) and completely replace the value.

If you leave “#INF” record in XML file, models for classification that will be created based on it will have “#IND” records and when you will try to run classification based upon these models – the process will be terminated complaining about unrecognised “#IND”-values, because values must be numeric and not characters.

I don’t know what causes the “#INF” records to appear in the first place, maybe not properly defined no-data values? In some cased it is impossible to provide correct no-data value due to limitations of the input field (I use OTB from within Processing module for QGIS).

After significant encoding handling changes in GDAL>=1.9 it became quite challenging to handle non-Latin attributes (Cyrillic in particular) stored in .dbf part of a shp-file. So the majority of the Cyrillic and other non-Latin users have to stick with GDAl 1.8.1 for now.

The cause of the issue is described here (Rus). Briefly: GDAL>=1.9 attemts to re-encode the .dbf attributes to UTF-8 on the basis of the LDID (Language Driver ID) written in .dbf header. But unfortunately LDID is usually missing, and in particular QGIS does not write it to the .dbf-file it creates. In case when LDID is missing, GDAL>=1.9 assumes that encoding of the .dbf-file is ISO8859_1 (Latin-1) which makes non-Latin characters unreadable.

The workaround I’m currently using is creating additional .cpg-file, that contains the ID of the encoding used. For example if encoding is Windows-1251, .cpg-file contains following record: “1251” (without quotes). When .cpg-file is present, GDAL>=1.9 + QGIS works just fine.

————————————————————————-

UPD: on some OS you will need to use ID from Additional ID column instead of Encoding ID column.

For Windows you may also try to use “unofficial” version of QGIS from here (with encoding issue solved). But it is possible that its installer is in Russian. 

UPD2: There is another workaround. You can open .dbf-file in Libre Office Calc (Open Office Calc) providing encoding needed and save it from there. This will write necessary header to .dbf-file and QGIS will open attributes correctly.  Note that this also will make fields names written in upper case.

UPD3: there is a plugin for encoding fixing available.

————————————————————————-

Here you are a table of the encoding IDs (taken from here):

Encoding ID Encodind name Additional ID Other names
1252 Western iso-8859-1
except when 128-159 is used, use “Windows-1252”
iso8859-1, iso_8859-1, iso-8859-1, ANSI_X3.4-1968, iso-ir-6, ANSI_X3.4-1986, ISO_646, irv:1991, ISO646-US, us, IBM367, cp367, csASCII, latin1, iso_8859-1:1987, iso-ir-100, ibm819, cp819, Windows-1252
20105 us-ascii us-acii, ascii
28592 Central European (ISO) iso-8859-2 iso8859-2, iso-8859-2, iso_8859-2, latin2, iso_8859-2:1987, iso-ir-101, l2, csISOLatin2
1250 Central European (Windows) Windows-1250 Windows-1250, x-cp1250
1251 Cyrillic (Windows) Windows-1251 Windows-1251, x-cp1251
1253 Greek (Windows) Windows-1253 Windows-1253
1254 Turkish (Windows) Windows-1254 Windows-1254
932 Japanese (Shift-JIS) shift_jis shift_jis, x-sjis, ms_Kanji, csShiftJIS, x-ms-cp932
51932 Japanese (EUC) x-euc-jp Extended_UNIX_Code_Packed_Format_for_Japanese, csEUCPkdFmtJapanese, x-euc-jp, x-euc
50220 Japanese (JIS) iso-2022-jp csISO2022JP, iso-2022-jp
1257 Baltic (Windows) Windows-1257 windows-1257
950 Traditional Chinese (BIG5) big5 big5, csbig5, x-x-big5
936 Simplified Chinese (GB2312) gb2312 GB_2312-80, iso-ir-58, chinese, csISO58GB231280, csGB2312, gb2312
20866 Cyrillic (KOI8-R) koi8-r csKOI8R, koi8-r
949 Korean (KSC5601) ks_c_5601 ks_c_5601, ks_c_5601-1987, korean, csKSC56011987
1255 (logical) Hebrew (ISO-logical) Windows-1255 iso-8859-8i
1255 (visual) Hebrew (ISO-Visual) iso-8859-8 ISO-8859-8 Visual, ISO-8859-8 , ISO_8859-8, visual
862 Hebrew (DOS) dos-862 dos-862
1256 Arabic (Windows) Windows-1256 Windows-1256
720 Arabic (DOS) dos-720 dos-720
874 Thai Windows-874 Windows-874
1258 Vietnamese Windows-1258 Windows-1258
65001 Unicode UTF-8 UTF-8 UTF-8, unicode-1-1-utf-8, unicode-2-0-utf-8
65000 Unicode UTF-7 UNICODE-1-1-UTF-7 utf-7, UNICODE-1-1-UTF-7, csUnicode11UTF7, utf-7
50225 Korean (ISO) ISO-2022-KR ISO-2022-KR, csISO2022KR
52936 Simplified Chinese (HZ) HZ-GB-2312 HZ-GB-2312
28594 Baltic (ISO) iso-8869-4 ISO_8859-4:1988, iso-ir-110, ISO_8859-4, ISO-8859-4, latin4, l4, csISOLatin4
28585 Cyrillic (ISO) iso_8859-5 ISO_8859-5:1988, iso-ir-144, ISO_8859-5, ISO-8859-5, cyrillic, csISOLatinCyrillic, csISOLatin5
28597 Greek (ISO) iso-8859-7 ISO_8859-7:1987, iso-ir-126, ISO_8859-7, ISO-8859-7, ELOT_928, ECMA-118, greek, greek8, csISOLatinGreek
28599 Turkish (ISO) iso-8859-9 ISO_8859-9:1989, iso-ir-148, ISO_8859-9, ISO-8859-9, latin5, l5, csISOLatin5

It is known that QGIS is painfully slow in redrawing canvas when at least one of the vector layers contains a lot of features. The obvious way to speed up the process is to forbid to render layer (or some of the features) at certain scale(s). Yes, it works, but what do we have to do, when we want to see the whole picture? To disable scale rendering and wait couple of minuets until the several hundreds of thousands features will be redrawn? Bo-o-oring!

The workaround that I use is not that elegant (as a rule based feature scaling for example), but it can be used in certain situations. When your vector layer is large but it is necessary to see all the features – start a WMS-server for this particular layer. Raster renders a way more quickly and will be generated by different software. Taking into account that the layer is on the same machine or in the local network the speed increase will be magnificent.

So you produce WMS-layer from the needed layer (lets call it parent-layer) by WMS-sever of choice (I use GeoServer because it has a straightforward GUI that allows you to set up server and WMS-layer in minuets), add it in QGIS and place it under the parent-layer. Then set visibility of the parent-layer to the larger scale (that is suitable for editing). Now you will be able to see the whole picture at the small scale, and rendering still will be fast. And because WMS-layer is produced from the same layer all of the committed changes to the parent-layer will appear at WMS-layer immediately.

And you will have another thing to decide: whether to create the same [complicated] style for WMS-layer as for parent layer or not. Well, I do not bother)))

Small Scale

Large scale