stackitem: limit JSONization nesting level

We can have very deep reference types and attempt to JSONize them can easily
lead to OOM (even though there are no recursive references inside). Therefore
we have to limit them. While regular ToJSON() is buffer size limited to
MaxSize, ToJSONWithTypes is not and limiting it to MaxSize output will require
substantial rewriting effort while not really providing fair result, MaxSize
is more about stack item size while its JSON representation can be much bigger
because of various overheads.

Initial thought was to limit it by element count based on
MaxIteratorResultItems, but the problem here is that we don't always have this
limit in contexts using ToJSONWithTypes (like notification event
marshaling). Thus we need some generic limit which would be fine for all
users.

We at the same time have maxJSONDepth used when deserializing from JSON, so
it can be used for marshaling as well, it's not often that we have deeper
structures in real results.

Inspired by neo-project/neo#2521.

pkg/vm/stackitem/json.go

@@ -221,6 +221,9 @@ func ToJSONWithTypes(item Item) ([]byte, error) {
 }
 
 func toJSONWithTypes(item Item, seen map[Item]bool) (interface{}, error) {
+	if len(seen) > maxJSONDepth {
+		return "", errors.New("too deep structure")
+	}
 	typ := item.Type()
 	result := map[string]interface{}{
 		"type": typ.String(),