For instance, if you want to specifically target a partition, it's possible by forcing the partition in the options: Advanced Options įor advanced usages, it's possible to alter more how the data are sent to Kafka. Extra columns will be ignoredĬoncerning Avro Records, we expect the JSON-ish representation (not bytes)īear in mind that the options and serializers of the underlying Producer Template will be applied (excepted the generation options).Ĭhoose a CSV file and click on Import to start producing to the topic. X columns with headers, containing at least a column with the "key" header and a column with the "value" header, in any order.a single column with no header (value only, keys will be set to null).two columns with no headers (the first being the key, the second being the value).It will open a dialog that explains the different formats accepted by Conduktor : On the producer view, after selecting a topic, the "PRODUCE FROM CSV" button (bottom right) gets enabled. This will create a new template in your producer, with the same deserialized payload, that you can resend at will! To do so, consume some data and open the details dialog, then click on the button "Copy To A Producer Template" This is useful to grab the initial payload and send it again and again after. When consuming records, it's possible to create a "Produce Template" from it. You may need to specify which type to use explicitely:ĭo not forget to click "SAVE" on the bottom right of the dialog to save your modification, Sometimes, it's not enough, when you have a union with many complex rypes. Unfortunately, when it comes to unions, it's a bit verbose to deal with.Ĭonduktor helps here by allowing simple JSON conversion. To produce Apache Avro data from JSON, Avro has a specific JSON encoding ( ). as time-micros: can be encoded with a string: "my_time": "02:47:41"."my_timestamp": " 01:03:29" (the default when consuming).as timestamp-millis and timestamp-micros: can be encoded with a string ISO 8601 or human friendly date:.Long : can be encoded with the "raw" long or a string as time-millis: can be encoded with a string: "my_time": "02:47:41".as date: can be encoded with a string: "my_date": "".Int: can be encoded with the "raw" int or a string The scale of the Apache Avro schema must fit or Conduktor will try to round it. eg: "name": "conduktor" should be written "name": "Y29uZHVrdG9y"īytes as a decimal logicalType: it can be encoded in JSON as a number or as a string.Here is the list of the Apache Avro types with logicalTypes and the special handling by Conduktor:īytes: when producing to a bytes field WITHOUT any logical type associated, it must be a base64 encoded string to avoid any weird/breaking/invisible characters that could cause parsing issues: Most types are simple and exist in JSON and Apache Avro (like integers, strings) but some needs a special handling by Conduktor to be "understood" and properly converted. When sending the data, Conduktor translates the JSON payload to an Apache Avro format. When producing data from Conduktor Desktop, the format must be in JSON. Representations of such complex types when consuming and when producing data to Avro topics. Apache Avro complex types (bytes, logical types.) Īpache Avro has special types that does not exist in plain JSON, such as "bytes", and its "logical types" which are encoded using a more simple type underneath, but have a particular semantics that we care about (decimal number, date, timestamp-micros etc.)Ĭonduktor Desktop supports Apache Avro 1.9.0 specification: Ĭonduktor Desktop is made for humans, therefore we handle human-friendly You can see the corresponding Apache Avro schema by clicking on "SCHEMA":Ĭlick "Send" (top right) as many times as you want to send the payload to Kafka.value: a JSON that Conduktor will convert to Apache Avro binary data using the selected "Topic Name Strategy", here "customer-orders-value".In this example, we are sending Apache Avro data corresponding to the schema of the topic: Avro: define the payload to send using JSON and Conduktor converts it to Apache Avro binary data automatically (keep reading to understand how special Apache Avro types are handled).JSON: it's a string but Conduktor validates that the payload is a proper JSON.You'll have a new dialog on top of Conduktor you can freely move around:įrom here, you must at least select which topic to send some data to, set the format of the key and value, and provide their payload (or empty to send a NULL).Ĭonduktor supports common Kafka serializers to produce data: How to Produce data? The Producer Dialog Ĭlick on the Producer button in the top bar to open the Producer dialog:
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |