Unicode storage of \u202b and \u202c in a Unicode-aware database?

I’m building a new product for toponyms and in it the Arabic shows kinda like this:

^IArabic^I<202b>ﺰﻤﺑﺎﺑﻮﻳ<202c>^I<202b>ﺞﻫﻭﺮﻳﺓ ﺰﻤﺑﺎﺑﻮﻳ<202c>$   

Actually not quite. This is a real cluster fuck for my ASCII-spewing terminal, so I’ll make an exception and screenshot text.

Arabic

My question is about those U202B “Right-To-Left Embedding”, and U202C “Pop Directional Formatting”. Do those get stored as data? My first assumption was that the characters were rendered and not in the file, but alas they are there..

360    5E03 97E6 5171 548C 56FD 000A 0009 0041 0072 0061 0062 0069 0063 0009 202B 0632    布韦共和国..Arabic..ز .............................................................................^HERE 389    0645 0628 0627 0628 0648 064A 202C 0009 202B 064F 062C 0647 0648 0631 064A 0629    مبابوي...ُجهورية .....................................^HERE.....^HERE 422    0020 0632 0645 0628 0627 0628 0648 064A 202C 000A 0009 004E 006F 0074 0065 0073    .زمبابوي...Notes ...............................................^HERE 

When storing Arabic in a database, do you typically store \u202b, and \u202c? They seem like they’re rendering characters and not technically data? I’m simply wanting to process this text to throw in a database, and wondering if these characters should be present in the database, or stripped before insert.

Background

  • The screenshot was taken with VIM in a terminal (Kitty) which does not support the Arabic text because all characters get displayed on a grid.
  • The text comes from text-extraction (using pdftotext)
  • The pdf was produced by the “United Nations Group of Experts on Geographical Names”. You can find the pdf (E/CONF.105/13) freely available here.

Can’t upgrade database from 2.2.7 to 2.3. Cannot process definition to array for type enum

Magento 2.2.7 to 2.3

Started at 2.2.3 and upgraded incrementally via command line.

Php 7.2 is enabled. Did not enable until ready to upgrade to 2.3 b/c 7.1 support ended on M2.3

SQL Version: 10.1.36-MariaDB-cll-lve

Quick history.

Tried to use Web Setup to go from 2.2.3 to 2.3. Failed with no failure screen. Tried again. Got a failure (I imagine after going thru this is was php 7.2 related and honestly, Web Setup shouldn’t have even allowed me the option to do it this way. CLI should be the only way) and provided rollback option, but didn’t know at this point what it would roll back to b/c it gave no indication on the prior failure what happened so I didn’t do the rollback. I think this may have be the source of my current database issues b/c it seems stuck in between states, as in between structure of 2.2.x and 2.3.

The Issue:

I upgraded via CLI to 2.2.7 and everything was working well except, as I noticed during the upgrade, I couldn’t make a DB backup due to “Area Code already existing”. I had an older backup, so I proceeded anyway. In too deep at this point.

I then tried to upgrade to 2.3 as per instructions at Magento Dev Docs. I made it to the end to rebuilt database via command

php bin/magento setup:upgrade

It failed with the following error:

Cannot process definition to array for type enum

It made it thru a few of the file system cleanups, then threw that error. So, I turned off the maintenance mode so I could see what was going on. Sure enough, the database didn’t upgrade. I thought it was due to the modules in the report at the end of this message being outdated, so I turned OFF ALL modules via CLI. I then tried to upgrade. Same “Cannot process definition to array for type enum” message.

At this point I am stumped and I’d really not like to roll back to the older db backup and lose some work if possible.

Any one have an idea on how I can get this database to complete the upgrade so I can enjoy the relative bliss that must be Magento 2.3? :smileylol:

Thanks in advance!

Kory

—–begin error report from post db upgrade——–

{“0″:”Please upgrade your database: Run \”bin/magento setup:upgrade\” from the Magento root directory.\nThe following modules are outdated:\nMSP_ReCaptcha schema: current version – none, required version – 1.5.0\nMSP_ReCaptcha data: current version – none, required version – 1.5.0\nMSP_TwoFactorAuth schema: current version – none, required version – 2.2.0\nMSP_TwoFactorAuth data: current version – none, required version – 2.2.0\nAmazon_Core schema: current version – 2.1.2, required version – 3.0.0\nAmazon_Core data: current version – 2.1.2, required version – 3.0.0\nMagento_Inventory schema: current version – none, required version – 1.0.0\nMagento_Inventory data: current version – none, required version – 1.0.0\nMagento_InventoryAdminUi schema: current version – none, required version – 1.0.0\nMagento_InventoryAdminUi data: current version – none, required version – 1.0.0\nMagento_InventoryApi schema: current version – none, required version – 1.0.0\nMagento_InventoryApi data: current version – none, required version – 1.0.0\nMagento_InventoryBundleProduct schema: current version – none, required version – 1.0.0\nMagento_InventoryBundleProduct data: current version – none, required version – 1.0.0\nMagento_InventoryBundleProductAdminUi schema: current version – none, required version – 1.0.0\nMagento_InventoryBundleProductAdminUi data: current version – none, required version – 1.0.0\nMagento_InventoryCatalog schema: current version – none, required version – 1.0.0\nMagento_InventoryCatalog data: current version – none, required version – 1.0.0\nMagento_InventorySales schema: current version – none, required version – 1.0.0\nMagento_InventorySales data: current version – none, required version – 1.0.0\nMagento_InventoryCatalogAdminUi schema: current version – none, required version – 1.0.0\nMagento_InventoryCatalogAdminUi data: current version – none, required version – 1.0.0\nMagento_InventoryCatalogApi schema: current version – none, required version – 1.0.0\nMagento_InventoryCatalogApi data: current version – none, required version – 1.0.0\nMagento_InventoryCatalogSearch schema: current version – none, required version – 1.0.0\nMagento_InventoryCatalogSearch data: current version – none, required version – 1.0.0\nMagento_InventoryConfigurableProduct schema: current version – none, required version – 1.0.0\nMagento_InventoryConfigurableProduct data: current version – none, required version – 1.0.0\nMagento_InventoryConfigurableProductAdminUi schema: current version – none, required version – 1.0.0\nMagento_InventoryConfigurableProductAdminUi data: current version – none, required version – 1.0.0\nMagento_InventoryConfigurableProductIndexer schema: current version – none, required version – 1.0.0\nMagento_InventoryConfigurableProductIndexer data: current version – none, required version – 1.0.0\nMagento_InventoryConfiguration schema: current version – none, required version – 1.0.0\nMagento_InventoryConfiguration data: current version – none, required version – 1.0.0\nMagento_InventoryConfigurationApi schema: current version – none, required version – 1.0.0\nMagento_InventoryConfigurationApi data: current version – none, required version – 1.0.0\nMagento_InventoryGroupedProduct schema: current version – none, required version – 1.0.0\nMagento_InventoryGroupedProduct data: current version – none, required version – 1.0.0\nMagento_InventoryGroupedProductAdminUi schema: current version – none, required version – 1.0.0\nMagento_InventoryGroupedProductAdminUi data: current version – none, required version – 1.0.0\nMagento_InventoryGroupedProductIndexer schema: current version – none, required version – 1.0.0\nMagento_InventoryGroupedProductIndexer data: current version – none, required version – 1.0.0\nMagento_InventoryImportExport schema: current version – none, required version – 1.0.0\nMagento_InventoryImportExport data: current version – none, required version – 1.0.0\nMagento_InventoryIndexer schema: current version – none, required version – 1.0.0\nMagento_InventoryIndexer data: current version – none, required version – 1.0.0\nMagento_InventoryLowQuantityNotification schema: current version – none, required version – 1.0.0\nMagento_InventoryLowQuantityNotification data: current version – none, required version – 1.0.0\nMagento_InventoryLowQuantityNotificationAdminUi schema: current version – none, required version – 1.0.0\nMagento_InventoryLowQuantityNotificationAdminUi data: current version – none, required version – 1.0.0\nMagento_InventoryLowQuantityNotificationApi schema: current version – none, required version – 1.0.0\nMagento_InventoryLowQuantityNotificationApi data: current version – none, required version – 1.0.0\nMagento_InventoryMultiDimensionalIndexerApi schema: current version – none, required version – 1.0.0\nMagento_InventoryMultiDimensionalIndexerApi data: current version – none, required version – 1.0.0\nMagento_InventoryProductAlert schema: current version – none, required version – 1.0.0\nMagento_InventoryProductAlert data: current version – none, required version – 1.0.0\nMagento_InventoryReservations schema: current version – none, required version – 1.0.0\nMagento_InventoryReservations data: current version – none, required version – 1.0.0\nMagento_InventoryReservationsApi schema: current version – none, required version – 1.0.0\nMagento_InventoryReservationsApi data: current version – none, required version – 1.0.0\nMagento_InventoryCache schema: current version – none, required version – 1.0.0\nMagento_InventoryCache data: current version – none, required version – 1.0.0\nMagento_InventorySalesAdminUi schema: current version – none, required version – 1.0.0\nMagento_InventorySalesAdminUi data: current version – none, required version – 1.0.0\nMagento_InventorySalesApi schema: current version – none, required version – 1.0.0\nMagento_InventorySalesApi data: current version – none, required version – 1.0.0\nMagento_InventoryShipping schema: current version – none, required version – 1.0.0\nMagento_InventoryShipping data: current version – none, required version – 1.0.0\nMagento_InventorySourceDeductionApi schema: current version – none, required version – 1.0.0\nMagento_InventorySourceDeductionApi data: current version – none, required version – 1.0.0\nMagento_InventorySourceSelection schema: current version – none, required version – 1.0.0\nMagento_InventorySourceSelection data: current version – none, required version – 1.0.0\nMagento_InventorySourceSelectionApi schema: current version – none, required version – 1.0.0\nMagento_InventorySourceSelectionApi data: current version – none, required version – 1.0.0\nKlarna_Core schema: current version – 4.5.2, required version – 5.0.0\nKlarna_Core data: current version – 4.5.2, required version – 5.0.0\nAmazon_Login schema: current version – 2.1.1, required version – 3.0.0\nAmazon_Login data: current version – 2.1.1, required version – 3.0.0\nAmazon_Payment schema: current version – 2.1.1, required version – 3.0.0\nAmazon_Payment data: current version – 2.1.1, required version – 3.0.0\nKlarna_Ordermanagement schema: current version – 4.4.1, required version – 5.0.2\nKlarna_Ordermanagement data: current version – 4.4.1, required version – 5.0.2\nMagento_InventoryShippingAdminUi schema: current version – none, required version – 1.0.0\nMagento_InventoryShippingAdminUi data: current version – none, required version – 1.0.0\nKlarna_Kp schema: current version – 5.5.1, required version – 6.0.0\nKlarna_Kp data: current version – 5.5.1, required version – 6.0.0\nDotdigitalgroup_Email schema: current version – 2.6.1, required version – 3.0.1\nDotdigitalgroup_Email data: current version – 2.6.1, required version – 3.0.1\nTemando_Shipping schema: current version – 1.2.1, required version – 1.4.0\nTemando_Shipping data: current version – 1.2.1, required version – 1.4.0″,”1″:”#0 /home/stebbins/public_html/vendor/magento/framework/Interception/Interceptor.php(121): Magento\Framework\Module\Plugin\DbStatusValidator->beforeDispatch(Object(Magento\Framework\App\FrontController\Interceptor), Object(Magento\Framework\App\Request\Http))\n#1 /home/stebbins/public_html/vendor/magento/module-page-cache/Model/App/FrontController/BuiltinPlugin.php(73): Magento\Framework\App\FrontController\Interceptor->Magento\Framework\Interception\{closure}(Object(Magento\Framework\App\Request\Http))\n#2 /home/stebbins/public_html/vendor/magento/framework/Interception/Interceptor.php(135): Magento\PageCache\Model\App\FrontController\BuiltinPlugin->aroundDispatch(Object(Magento\Framework\App\FrontController\Interceptor), Object(Closure), Object(Magento\Framework\App\Request\Http))\n#3 /home/stebbins/public_html/vendor/magento/framework/Interception/Interceptor.php(153): Magento\Framework\App\FrontController\Interceptor->Magento\Framework\Interception\{closure}(Object(Magento\Framework\App\Request\Http))\n#4 /home/stebbins/public_html/generated/code/Magento/Framework/App/FrontController/Interceptor.php(26): Magento\Framework\App\FrontController\Interceptor->___callPlugins(‘dispatch’, Array, Array)\n#5 /home/stebbins/public_html/vendor/magento/framework/App/Http.php(135): Magento\Framework\App\FrontController\Interceptor->dispatch(Object(Magento\Framework\App\Request\Http))\n#6 /home/stebbins/public_html/generated/code/Magento/Framework/App/Http/Interceptor.php(24): Magento\Framework\App\Http->launch()\n#7 /home/stebbins/public_html/vendor/magento/framework/App/Bootstrap.php(258): Magento\Framework\App\Http\Interceptor->launch()\n#8 /home/stebbins/public_html/index.php(39): Magento\Framework\App\Bootstrap->run(Object(Magento\Framework\App\Http\Interceptor))\n#9 {main}”,”url”:”/”,”script_name”:”/index.php”}

Designing database job monitoring

I have several Oracle database where my in-house applications are running. Those applications use both dba_jobs and dba_scheduler_jobs.

I want to write monitoring function: check_my_jobs which will be called periodically by Nagios to check if everything is OK with my jobs. (Are they running? Is it Broken? Is next_run_date delayed? and so on)

Solutions:

Due to the fact that I have to monitor jobs on different databases, there is two way of implementing the solution:

  1. Create monitoring function and configuration tables only on one database which will check jobs on every database using the Database link.

    pros: Centralized functionality, easy to maintain.
    cons: I have to make a check using the Database link.

  2. Create monitoring function and configuration tables in every database where I want to check jobs.

    pros: I don’t have to use DB link
    cons: Duplicated monitoring code on every database

Which solution is better?

In a graph database, how do I know whether to use a node, node property or relationship property?

For example, given node types User and Post, and expressing that a user posted something on Twitter, what factors would I need to consider in deciding between:

(u:User)-[posted]->(p:Post {platform: 'twitter'}),

(u:User)-[posted {platform: 'Twitter'}]->(p:Post)

and

(u:User)-[posted]->(p:Post)-[postedOn]->(twitter:Platform)

It seems like each approach is valid but likely optimal under different circumstances.

Specifically, I’m using Neo4j in this case.

How to get username and password variables from a MySql Database for an IOS app?

I started IOS programming a few weeks ago and I’m trying to create a login page that pulls an username and password from a database that I’ve already created. I’ve set up the php file and linked it up to server and the info prints correctly. The problem I’m trying to solve is how to pull the data from the actual app(the Xcode Swift file).

I have been looking on the internet for a few days and the only thing I can find is how to print the data and not how to store it into a variable to cross-check with the user’s username and password.

My PHP File:

<?php  // Create connection $  con=mysqli_connect("localhost","username","password","dbname");  // Check connection if (mysqli_connect_errno())         { echo "Failed to connect to MySQL: " . mysqli_connect_error(); }      // This SQL statement selects ALL from the table 'Locations' $  sql = "SELECT * FROM Locations";  // Check if there are results if ($  result = mysqli_query($  con, $  sql))     { // If so, then create a results array and a temporary one // to hold the data $  resultArray = array(); $  tempArray = array();  // Loop through each row in the result set while($  row = $  result->fetch_object()) {     // Add each row into our results array     $  tempArray = $  row;     array_push($  resultArray, $  tempArray); }  // Finally, encode the array to JSON and output the results echo json_encode($  resultArray); }  // Close connections mysqli_close($  con); ?> 

Could someone guide me in the right direction to get the data from the server and store it into a variable in swift code?

Thanks in advance!

What database would be ideal for storing hierarchical data, calculating rollups and paginating through them?

I’m working on the data model for a project that needs to store hierarchical data (similar to employee-supervisor relationship). The depth of the tree is unknown but usually less than 15.

The two primary use-cases are:

  1. Count the number of children (direct and indirect) under a node in real-time. Also needs the ability to filter the count based on additional criteria.
  2. Paginate through the actual records that make up the count. Also needs the ability to filter the results.

The size of the dataset is millions of records, but will never grow to billions of records.

What we’ve looked at:

Graph DB:

  • Since the data is essentially a tree.
  • It works pretty well for the 1st use-case. Latency is in milliseconds for almost the entire tree (except the root and one level below, which is in seconds).
  • Not sure how efficient pagination would be for a graph DB.

Relational:

  • Postgres offers pretty similar performance to the graph DB with its recursive queries.
  • Works well for the second use-case too.
  • Not sure about scalability. Trend within company is to move away from Relational DB.

ElasticSearch

  • Won’t work for the first use-case (unless I use a materialized path, which may not work here since the hierarchy may change)
  • Works for the second use-case

Streams

  • For the first case, I could enable streaming and pre-calculate the counts, but I lose the filtering ability.

Am I missing out on any good candidates? Also, would it be better to use two separate solutions for the two use-cases?

Command Line Utility For Browsing Apple’s Photo Database Like a Filesystem

Is there a command-line utility to allow browsing and editing photos in the Apple Photos database like you would browse a file system? The Photos app is really terrible when trying to perform operations on large data sets (in my case, delete all of the thumbnails that I accidentally imported) as it foolishly tries to load all of the photo metadata into memory when making selections of matching files to perform any select and this brings my computer to a halt as it allocates 101 GB of virtual memory and it just sits there paging for hours. I have tried to use third party “deduplication” tools but they are all terrible. I really just want to perform some basic operations on them using tools like exiv2 and GraphicsMagick to write some shell scripts to do get what I need done so I don’t have to put up with Apple’s nonsense with Photos. Any ideas? I have not found anything yet. Honestly, a Fuse plugin would be ideal, but alas- such a thing does not seem to exist.

RESTful CRUD API and Database Design/Structure for specific project

I need help designing a backend-service that consists of an RESTful CRUD API in NodeJS and a MongoDB (Document-based).

The final product shall work as an unilateral „chat“ service.

There are two types of users, let‘s say teachers and parents. The teachers shall be able to announce upcoming events, write letters etc. All announcements/letters/… target a specific course (class; f.ex. Chem.104 or Math.457).

The parents shall then be able to subscribe to some of the courses (classes) (usually, they subscribe to exactly those courses that their children are in).

In sum, parents won‘t be allowed to write any messages to anyone, whereas teachers can write messages to courses/classes that parents may subscribe to.

Our initial attempt or our thoughts were to create group-logins; so that every course would receive an own password and username with which all parents would be able to login. The advantage with this strategy (instead of a single-user-auth) is that we don‘t need to care about deleting accounts when students leave school.

Now, what endpoints shall I use for the API? I‘m generally new to API design, so I posted my question here looking for some experience.

In addition, I‘m wondering how exactly a user model for the DB shall loook like – do we need user roles to determine whether the user is a teacher or parent? Remember, a parent will never „upgrade“ his account to a teacher.

This question does not expect an answer that goes into coding details, as it would be way too broad then. Instead, I am looking for the basic API design including the endpoints and DB storage possibilities.

Magento 2 upgradeSchema.php Not Adding the Column on Database

I am trying to add column ‘address’ on existing upgradeSchema.php, but it not worked.

Here is my code:

<?php      namespace Testing\Test\Setup;       use Magento\Framework\DB\Ddl\Table;     use Magento\Framework\Setup\UpgradeSchemaInterface;     use Magento\Framework\Setup\ModuleContextInterface;     use Magento\Framework\Setup\SchemaSetupInterface;      class UpgradeSchema implements UpgradeSchemaInterface     {          public function upgrade(SchemaSetupInterface $  setup, ModuleContextInterface $  context)         {             $  setup->startSetup();              if (version_compare($  context->getVersion(), '1.0.1', '<')) {                 $  setup->getConnection()->addColumn(                     $  setup->getTable('testing_test_item'),                     'description',                     [                         'type' => Table::TYPE_TEXT,                         'nullable' => true,                         'comment' => 'Item Description'                     ]                 );                  $  setup->getConnection()->addColumn(                     $  setup->getTable('testing_test_item'),                     'address',                     [                         'type' => Table::TYPE_TEXT,                         'nullable' => true,                         'comment' => 'Item Address'                     ]                 );             }             $  setup->endSetup();         }     }     ?> 

PROBLEM WITH COLLATION (NULL) CREATING A DATABASE SQL SERVER

I am creating a DATABASE with the next command CREATE DATABASE CARGADOR COLLATE SQL_Latin1_General_CP1_CI_AS GO When I EXECUTE THE NEXT COMMAND select [Collation] = DatabasePropertyEx(‘CARGADOR’,’Collation’) THE RESULT IS

Collation NULL

The question is why is null

The result of the next command

SELECT SERVERPROPERTY(‘collation’) SQLServerCollation ,DATABASEPROPERTYEX(‘master’, ‘Collation’) AS MasterDBCollation

SQLServerCollation MasterDBCollation Modern_Spanish_CI_AS Modern_Spanish_CI_AS

AND SELECT @@VERSION

Microsoft SQL Server 2016 (SP1-GDR) (KB4458842) – 13.0.4224.16 (X64) Aug 18 2018 09:00:06 Copyright (c) Microsoft Corporation Express Edition (64-bit) on Windows 10 Home Single Language 6.3 (Build 17134: )

FIRST OF ALL, THANKS