How can I create a json in powershell with an array of objects that has a nested array of objects [duplicate] - arrays

Why do I get unexpected ConvertTo-Json results?
Meta issue
Stackoverflow has a good mechanism to prevent duplicate questions but as far as I can see there is no mechanism to prevent questions that have a duplicate cause. Take this question as a an example: almost every week a new question comes in with the same cause, yet it is often difficult to define it as a duplicate because the question itself is just a slightly different.
Nevertheless, I wouldn't be surprised if this question/answer itself ends up as a duplicate (or off-topic) but unfortunately stackoverflow has no possibility to write an article to prevent other programmers from continuing writing questions caused by this “known” pitfall.
Duplicates
A few examples of similar questions with the same common cause:
PowerShell ConvertTo-Json does not convert Array as expected
(yesterday)
Powershell ConvertTo-json with embedded hashtable
powershell “ConvertTo-Json” has messed json format output
Nested arrays and ConvertTo-Json
Powershell ConvertTo-JSON missing nested level
How to save a JSON object to a file using Powershell?
Cannot convert PSCustomObjects within array back to JSON correctly
ConvertTo-Json flattens arrays over 3 levels deep
Add an array of objects to a PSObject at once
Why does ConvertTo-Json drop values
…
Different
So, were does this “self-answered” question differ from the above duplicates?
It has the common cause in the title and with that it might better prevent repeating questions due to the same cause.

Answer
ConvertTo-Json has a -Depth parameter:
Specifies how many levels of contained objects are included in the
JSON representation.
The default value is 2.
TL;DR
Probably because ConvertTo-Json terminates branches that are deeper than the default -Depth (2) with a (.Net) full type name, programmers assume a bug or a cmdlet limitation and do not read the help or about.
Personally, I think a string with a simple ellipsis (three dots: …) at the end of the cut off branch, would have a clearer meaning (see also: Github issue: 8381)
Why?
This issue often ends up in another discussion as well: Why is the depth limited at all?
Some objects have circular references, meaning that a child object could refer to a parent (or one of its grandparents) causing a infinitive loop if it would be serialized to JSON.
Take for example the following hash table with a parent property that refers to the object itself:
$Test = #{Guid = New-Guid}
$Test.Parent = $Test
If you execute: $Test | ConvertTo-Json it will conveniently stop at a depth level of 2 by default:
{
"Guid": "a274d017-5188-4d91-b960-023c06159dcc",
"Parent": {
"Guid": "a274d017-5188-4d91-b960-023c06159dcc",
"Parent": {
"Guid": "a274d017-5188-4d91-b960-023c06159dcc",
"Parent": "System.Collections.Hashtable"
}
}
}
This is why it is not a good idea to automatically set the -Depth to a large amount.

Your question clearly illustrates how much of a pain point the current default ConvertTo-Json behavior is.
As for the justification of the behavior:
Some objects have circular references
Internally, ConvertTo-Json actually has a safety feature that prevents infinitely deep serialization: it reports an error if the depth exceeds 100 levels.
This internal limit should be enough.
While -Depth can be useful to intentionally truncate an input object tree whose full depth you don't need, -Depth defaulting to 2 and quietly truncating the output amounts to quiet de-facto failure of the serialization from the unsuspecting user's perspective - failure that may not be discovered until later.
The seemingly arbitrary and quiet truncation is surprising to most users, and having to account for it in every ConvertTo-Json call is an unnecessary burden.
I've created this GitHub issue to request changing the current behavior, specifically as follows:
Remove the default value for -Depth
The hard-coded internal limit of 100, which when exceeded, reports an error, is sufficient to prevent "runaway" JSON strings stemming from object tress with cyclic dependencies.
Typical input objects will then be fully serialized by default, which is typically the intent.
Use -Depth solely at the user's discretion in order to:
Intentionally truncate the input object tree at the specified depth.
On rare occasions, allow serialization of object trees that are deeper than 100 levels.
Make your voice heard there, if you'd like to see this change happen (or disagree).

Related

Sorting and managing numerous variables

My project has classes which, unavoidably, contain hundreds upon hundreds of variables that I'm always having to keep straight. For example, I'm always having to keep track of specific kinds of variables for a recurring set of "items" that occur inside of a class, where placing those variables between multiple classes would cause a lot of confusion.
How do I better sort my variables to keep from going crazy, especially when it comes time to save my data?
Am I missing something? Actionscript is an Object Oriented language, so you might have hundreds of variables, but unless you've somehow treated it like a grab bag and dumped it all in one place, everything should be to hand. Without knowing what all you're keeping track of, it's hard to give concrete advice, but here's an example from a current project I'm working on, which is a platform for building pre-employment assessments.
The basic unit is a Question. A Question has a stem, text that can go in the status bar, a collection of answers, and a collection of measures of things we're tracking about what the user does in that particular type of questions.
The measures are, again, their own type of object, and come in two "flavors": one that is used to track a time limit and one that isn't. The measure has a name (so we know where to write back to the database) and a value (which tells us what). Timed ones also have a property for the time limit.
When we need to time the question, we hand that measure to yet another object that counts the time down and a separate object that displays the time (if appropriate for the situation). The answers, known as distractors, have a label and a value that they can impart to the appropriate measure based on the user selection. For example, if a user selects "d", its value, "4" is transferred to the measure that stores the user's selection.
Once the user submits his answer, we loop through all the measures for the question and send those to the database. If those were not treated as a collection (in this case, a Vector), we'd have to know exactly what specific measures are being stored for each question and each question would have a very different structure that we'd have to dig through. So if looping through collections is your issue, I think you should revisit that idea. It saves a lot of code and is FAR more efficient than "var1", "var2", "var3."
If the part you think is unweildy is the type checking you have to do because literally anything could be in there, then Vector could be a good solution for you as long as you're using at least Flash Player 10.
So, in summary:
When you have a lot of related properties, write a Class that keeps all of those related bits and pieces together (like my Question).
When objects have 0-n "things" that are all of the same or very similar, use a collection of some sort, such as an Array or Vector, to allow you to iterate through them as a group and perform the same operation on each (for example, each Question is part of a larger grouping that allows each question to be presented in turn, and each question has a collection of distractors and another of measures.
These two concepts, used together, should help keep your information tidy and organized.
While I'm certain there are numerous ways of keeping arrays straight, I have found a method that works well for me. Best of all, it collapses large amounts of information into a handful of arrays that I can parse to an XML file or other storage method. I call this method my "indexed array system".
There are actually multiple ways to do this: creating a handful of 1-dimensional arrays, or creating 2-dimensional (or higher) array(s). Both work equally well, so choose the one that works best for your code. I'm only going to show the 1-dimensional method here. Those of you who are familiar with arrays can probably figure out how to rewrite this to use higher dimensional arrays.
I use Actionscript 3, but this approach should work with almost any programming or scripting language.
In this example, I'm trying to keep various "properties" of different "activities" straight. In this case, we'll say these properties are Level, High Score, and Play Count. We'll call the activities Pinball, Word Search, Maze, and Memory.
This method involves creating multiple arrays, one for each property, and creating constants that hold the integer "key" used for each activity.
We'll start by creating the constants, as integers. Constants work for this, because we never change them after compile. The value we put into each constant is the index the corresponding data will always be stored at in the arrays.
const pinball:int = 0;
const wordsearch:int = 1;
const maze:int = 2;
const memory:int = 3;
Now, we create the arrays. Remember, arrays start counting from zero. Since we want to be able to modify the values, this should be a regular variable.
Note, I am constructing the array to be the specific length we need, with the default value for the desired data type in each slot. I've used all integers here, but you can use just about any data type you need.
var highscore:Array = [0, 0, 0, 0];
var level:Array = [0, 0, 0, 0];
var playcount:Array = [0, 0, 0, 0];
So, we have a consistent "address" for each property, and we only had to create four constants, and three arrays, instead of 12 variables.
Now we need to create the functions to read and write to the arrays using this system. This is where the real beauty of the system comes in. Be sure this function is written in public scope if you want to read/write the arrays from outside this class.
To create the function that gets data from the arrays, we need two arguments: the name of the activity and the name of the property. We also want to set up this function to return a value of any type.
GOTCHA WARNING: In Actionscript 3, this won't work in static classes or functions, as it relies on the "this" keyword.
public function fetchData(act:String, prop:String):*
{
var r:*;
r = this[prop][this[act]];
return r;
}
That queer bit of code, r = this[prop][this[act]], simply uses the provided strings "act" and "prop" as the names of the constant and array, and sets the resulting value to r. Thus, if you feed the function the parameters ("maze", "highscore"), that code will essentially act like r = highscore[2] (remember, this[act] returns the integer value assigned to it.)
The writing method works essentially the same way, except we need one additional argument, the data to be written. This argument needs to be able to accept any
GOTCHA WARNING: One significant drawback to this system with strict typing languages is that you must remember the data type for the array you're writing to. The compiler cannot catch these type errors, so your program will simply throw a fatal error if it tries to write the wrong value type.
One clever way around this is to create different functions for different data types, so passing the wrong data type in an argument will trigger a compile-time error.
public function writeData(act:String, prop:String, val:*):void
{
this[prop][this[act]] = val;
}
Now, we just have one additional problem. What happens if we pass an activity or property name that doesn't exist? To protect against this, we just need one more function.
This function will validate a provided constant or variable key by attempting to access it, and catching the resulting fatal error, returning false instead. If the key is valid, it will return true.
function validateName(ID:String):Boolean
{
var checkthis:*
var r:Boolean = true;
try
{
checkthis = this[ID];
}
catch (error:ReferenceError)
{
r = false;
}
return r;
}
Now, we just need to adjust our other two functions to take advantage of this. We'll wrap the function's code inside an if statement.
If one of the keys is invalid, the function will do nothing - it will fail silently. To get around this, just put a trace (a.k.a. print) statement or a non-fatal error in the else construct.
public function fetchData(act:String, prop:String):*
{
var r:*;
if(validateName(act) && validateName(prop))
{
r = this[prop][this[act]];
return r;
}
}
public function writeData(act:String, prop:String, val:*):void
{
if(validateName(act) && validateName(prop))
{
this[prop][this[act]] = val;
}
}
Now, to use these functions, you simply need to use one line of code each. For the example, we'll say we have a text object in the GUI that shows the high score, called txtHighScore. I've omitted the necessary typecasting for the sake of the example.
//Get the high score.
txtHighScore.text = fetchData("maze", "highscore");
//Write the new high score.
writeData("maze", "highscore", txtHighScore.text);
I hope ya'll will find this tutorial useful in sorting and managing your variables.
(Afternote: You can probably do something similar with dictionaries or databases, but I prefer the flexibility with this method.)

Quagga Javascript Barcode Scanner

Is there a way to scan Code 39 and UPC at the same time without changing the barcode type from the options? I tried adding both of them to active barcodes list. Still without changing the option to UPC or Code 39, it is not scanning
I left you some questions with the details you should have given so people can help you more easily but I'll try to answer what I think that your question is. Not sure.
I think you ask why you need to set the reader engine before reading a barcode. I just read the docs for a minute and it is answered there:
Inside the decoder section:
"The most important property is readers which takes an array of types of barcodes which should be decoded during the session.
(...)
Why are not all types activated by default? Simply because one
should explicitly define the set of barcodes for their use-case. More
decoders means more possible clashes, or false-positives. One should
take care of the order the readers are given, since some might return
a value even though it is not the correct type (EAN-13 vs. UPC-A).
The multiple property tells the decoder if it should continue decoding
after finding a valid barcode. If multiple is set to true, the results
will be returned as an array of result objects. Each object in the
array will have a box, and may have a codeResult depending on the
success of decoding the individual box."
You need to use a config like this one with several readers:
{
readers: [
'code_39_reader',
'code_39_vin_reader',
'upc_reader',
'upc_e_reader'
]
multiple: false
}

What is the technical term for a value which can be either a scalar or an array?

I'm working with the JSON API format, which has the notion of a data property which can hold either a scalar (single) or array (multiple) value. I'm writing code for encoding and decoding into the format, and when naming my types, was trying to come up with a good name for such types of values. In TypeScript, it would be
type Poly<T> = T | T[];
For your information, here is the relevant part of the JSON API doc (my emphasis):
Primary data MUST be either:
a single resource object, a single resource identifier object, or null, for requests that target single resources
an array of resource objects, an array of resource identifier objects, or an empty array ([]), for requests that target resource collections
As an example, here is a mapping function for such mutant values:
function polymap<T, U>(data: Poly<T>, fn: (input: T) => U, thisArg?: any): Poly<U> {
if (data instanceof Array) return (data as T[]).map(fn, thisArg);
return fn.call(thisArg, data as T);
}
Anyway, as you can see, I'm going with "poly", but is there any established terminology for this, or other good suggestions?
First, the difference between scalars and arrays isn't the number of elements, it's the dimensionality.
Scalars are arrays. Specifically, they're 0-dimensional arrays. So you'd just call all of them arrays.
But note that usually the focus isn't on what values the variable can hold, but what operations are allowed on the variable's potential values.
Some operations can generalize from 1 element to N elements, which seems to be what you want.
The CS-y term for this kind of operation is a "vectorizing operation".
The math term for this kind of operation is a "lifting operation".
I've never heard of anything like this and judging from the fact that the available JSON data types shown here
http://www.tutorialspoint.com/json/json_data_types.htm do not mention anything related to such a data type, I'm betting that this is just smoke and mirrors implemented behind the scenes.
Scalar values and arrays are structured very differently. Combining the two into a true data type would be self contradicting. I'm betting that when the variable is instantiated a method is called somewhere to check the value for an array or scalar value, at which point one of two things takes place. 1- The data type is automatically set as an array and if a scalar is given, it is converted to an array of length 1 and whenever it is called behind the scenes an array is being accessed at index 0. Or 2- The behind the scenes method checks what the data type that was passed in is and sets the same data type for the variable given before instantiation actually takes place.
If a MonoPoly Mohn-O-Pohl-E (if you will because I can't find anything else to call it and it looks like Monopoly) was a scalar and is set to an array data type at some point, the old value can be destroyed and a new one assigned with the same name but as an array. This can happen vise versa and all this can be done behind the scenes as well, making this data type appear to house the description of a scalar along with an array.
is there any established terminology for this, or other good
suggestions?
I'm going to ignore "established terminology", and likely "good" as well, and make the following suggestions on a linguistic basis. My personal choice, due to its sense of one or more rungs and its being the latin origin of scalar, would be:
ladder
Another possible noun:
assemblage
Like constant, I think the following adjectives are ripe to become nouns:
inconstant (similar to incontinent, "insufficient voluntary control" which seem to be what you described you have over this API issue.)
transferable
indiscrete (nicely homophonic to indiscreet)
dual-purpose
multipurpose (the adjective our schools love)
This might even be an oportunity to coin a new word:
polyunary
versutility (my second favorite)
Such functionality is likely replacing a scalar with an array when another element is "added" to a slot. AFAIK there's no term for this, so the question seems more like an English language question.
Try these:
elastic scalar
expandable scalar
scarray (scalar/array)
scalarray (another portmanteau)
arrayable
tardis (holds more than it appears to)

Php prefix tree implementation versus assoc array [closed]

UPD: I moved original question to https://codereview.stackexchange.com/questions/127055/building-tree-graph-from-dictionary-performance-issues
Here is a short version, without codes.
I'm trying to build a prefix tree from dictionary. So, using the following dictionary 'and','anna','ape','apple', graph should look like this:
I've tried 2 approaches: using associative array and using self-written tree/node classes.
Note: original dictionary is something about 8 MB and contains >600000 words.
Question: is there any good(fast/efficient) way to do it?
I've tried so far:
php associative arrays (they are not very flexible for future work with this graph).
self-written Tree/Node classes (performance issues - execution time rises by up to 7x, memory usage rises by 2x even without implementing anything except just inserting function).
Sample codes are available on codereview (the very first link in question)
As long as I've switched to C++ and got a good answer on codereview, I'll just answer my own question here.
There is one more way to do it way more time-efficient by increasing memory usage(it's not really big increase, compared to "array of arrays of arrays..." approach). The approach is called "double array trie" and you can read info on this topic here and read the aforementioned answer on codereview to see an example of implementation.
It's more time-efficient, yet it allows less flexibility/convenience for future trie use (compared to OOP approach).
So the final answer on this question for me is: "php is not the best tool to work with really big tries with".

Are there limitations to limit to the three.js userData property?

I am using the Three.js Object3D userData property to store information from a MySQL database serialized into json pairs to give me data to perform various actions when selecting objects which represent saved data. It seems to work nicely for a few pairs.
I note from the reference a warning to not to store references to functions as they will not be cloned. Can anyone tell me if there any other limitations to this property (number of pairs, hierarchical data, etc.)? I want to store 2-3000 words of text, images, blobs etc., but prefer to ask over trial and error. the documents are a little sparse on such matters.
Many thanks... James
No there are not special limitations. It is simply a Javascript object:
https://github.com/mrdoob/three.js/blob/0fbc8afb348198e4924d9805d1d4be5869264418/src/core/Object3D.js#L85
this.userData = {};
So while your object is in memory, you can put any Javascript variables there. Only limitations are what you always have, the available memory basically. As Javascript objects can contain any types and hierarchy so you're off fine there.
I used this search to check this in the codebase: https://github.com/mrdoob/three.js/search?utf8=%E2%9C%93&q=userdata

Resources