Test FireWall Opening With Powershell

October 14, 2022

PowerShell Scripting

Read in minutes

With complex infrastructure, firewalls are often bothersome.
Is it our application or the network?
Powershell TcpListener class can help us.

Here is a sample script opening a specific port and printing incoming data along with connection information.

The Code
#region Hardcoded Parameters
$iPortToTest = 6666
#endregion
#region Internal Parameters
$_Listener
#endregion
#region Internal Function
Function Open-TCPPort {
    [CmdletBinding()] 
    Param ( 
        [Parameter(Mandatory=$true, Position=0)]
        [ValidateNotNullOrEmpty()] 
        [int] $Port
    )
    Process {
        Try {
            # Start Listener 
            $endpoint = new-object System.Net.IPEndPoint([ipaddress]::any,$Port)
            $listener = new-object System.Net.Sockets.TcpListener $endpoint
            $listener.start() 
            Write-Verbose ("[$(Get-Date)] Listener started on port {0} " -f $Port) -Verbose
            Return $listener
        }
        Catch {              
			$mess = "Couldn't start listener : " + $Error[0]
            Write-Error -Message $mess -ErrorAction Stop 
        }
    }
}
Function Receive-TCPMessage {
    [CmdletBinding()] 
    Param ( 
        [Parameter(Mandatory=$true, Position=0)]
        [ValidateNotNullOrEmpty()] 
        [System.Net.Sockets.TcpListener] $listener
    ) 
    Process {
        Try { 
 
            # Accept connection 
            $data = $listener.AcceptTcpClient()
            Write-Verbose ("[$(Get-Date)] New Connection from {0} Source port <{1}>" -f
            $data.Client.RemoteEndPoint.Address, $Data.Client.RemoteEndPoint.Port) -Verbose
        
            # Stream setup
            $stream = $data.GetStream() 
            $bytes = New-Object System.Byte[] 1024
            # Read Data from stream and write it to host
            while (($i = $stream.Read($bytes,0,$bytes.Length)) -ne 0){
                $EncodedText = New-Object System.Text.ASCIIEncoding
                $data = $EncodedText.GetString($bytes,0, $i)
                Write-Output $data
            }
         
        }
        Catch {
            $mess = "Receive Message failed with: `n" + $Error[0]
            Write-Error -Message $mess -ErrorAction Stop 
        }
        Finally {
            # Close stream
            $stream.close()
        }
    }
}
#endregion  Internal Function
#----------------------------------------------
#region Main Block
#----------------------------------------------
$_Listener = Open-TCPPort($iPortToTest)
try{
    while($true) {
        try{
            # Test for pending connection
            if($_Listener.Pending()){
                Receive-TCPMessage($_Listener)
            }
            Start-Sleep -Milliseconds 50
        }
        catch {        
            $mess = "Receive Message failed with: `n" + $Error[0]
            Write-Error -Message $mess -ErrorAction Stop 
        }
    }
}
finally{
         $_Listener.stop()
}
#----------------------------------------------
#endregion Main Block
#---------------------------------------------- 

The usage

Set the port you want to listen to in script parameter: $iPortToTest.

Run the script. A message should display the port is now listening.

Start Port Listening

Connection request will be displayed (machine, source port and incoming data).

A simple test, in local using putty:

Telnet With Putty
Connection successful
Data Received

SHARE ON :



comments

Purge ODI Logs through Database

May 29, 2020

Business Inteligence Data Integration

Read in 3 minutes

If you generate a lot of logs in ODI, purging through ODI built-in mechanism can be very slow. A lot faster to do it through Database, but you have to respect foreign keys. Here is a sample plsql script to do so.

Here is a simple script with one parameter which is the number of days of log you want to keep, it will there retrieve session number and delete in the logs table following the dependencies.

SHARE ON :


Related articles

March 25, 2024

Read in minutes

Getting started with the new Power BI Visual Calculations feature!

Power BI’s latest feature release, Visual Calculations, represents a paradigm shift in how users interact with data.      Rolled ...

February 20, 2024

Read in 5 minutes

Revolutionizing Data Engineering: The Power of Databricks’ Delta Live Tables and Unity Catalog

Databricks has emerged as a pivotal platform in the data engineering landscape, offering a comprehensive suite of tools designed to tackle the complexities of d...

November 20, 2023

Read in 5 minutes

AKABI’s Consultants Share Insights from Dataminds Connect 2023

Dataminds Connect 2023, a two-day event taking place in the charming city of Mechelen, Belgium, has proven to be a cornerstone in the world of IT and Microsoft ...


comments

Clean ODI Scenario with Groovy

September 27, 2019

Business Inteligence Data Integration

Read in 1 minutes

You may generate a lot of scenarii when developping ODI projet. When promoting, commiting to git… …you are usually only interested in the last functionnal scenario.

All the past being stored in git or promoted, you may like to clear all all scenarii. If yes, this groovy script may help you. It will delete all scenarii but the last version (sorted by version name, take care…).

The code has two parameters, the project code and a pattern for the package. May help you target specific scenario.

//Imports core
import oracle.odi.core.persistence.transaction.ITransactionDefinition;
import oracle.odi.core.persistence.transaction.support.DefaultTransactionDefinition;
import oracle.odi.core.persistence.transaction.ITransactionManager;
import oracle.odi.core.persistence.transaction.ITransactionStatus;

//Imports odi Objects
import oracle.odi.domain.project.OdiPackage;
import oracle.odi.domain.runtime.scenario.OdiScenario;
import oracle.odi.domain.project.finder.IOdiPackageFinder;
import oracle.odi.domain.runtime.scenario.finder.IOdiScenarioFinder;


// Parameters -- TO FILL --
String sourceProjectCode = 'MY_PROJECT_CODE';
String sourcePackageRegexPattern = '*';


println "    Start Scenarios Deletion";
println "-------------------------------------";

//Setup Transaction
ITransactionDefinition txnDef = new DefaultTransactionDefinition();
ITransactionManager tm = odiInstance.getTransactionManager();
ITransactionStatus txnStatus = tm.getTransaction(txnDef);

int scenarioDeletedCounter = 0;

try {
  //Init Scenario Finder
  IOdiScenarioFinder odiScenarioFinder = (IOdiScenarioFinder)odiInstance.getTransactionalEntityManager().getFinder(OdiScenario.class);
  //Loops through all packages in target project/fodlers
  for (OdiPackage odiPackageItem : ((IOdiPackageFinder)odiInstance.getTransactionalEntityManager().getFinder(OdiPackage.class)).findByProject(sourceProjectCode)){
    // Only generate Scenario for package matching pattenr
    if (!odiPackageItem.getName().matches(sourcePackageRegexPattern)) {
      continue;    
    }
    println "Deleting Scenarii for Package " + odiPackageItem.getName();
    
    odiScenCollection = odiScenarioFinder.findBySourcePackage(odiPackageItem.getInternalId());
    maxOdiScen = odiScenCollection.max{it.getVersion()};
    if (maxOdiScen != null) {
      for (OdiScenario odiscen : odiScenCollection ) {
        if (odiscen != maxOdiScen){
          println "Deleting Scenari "+ odiscen.getName() + " " + odiscen.getVersion();
          odiInstance.getTransactionalEntityManager().remove(odiscen);
          scenarioDeletedCounter ++;
        }
      }
    }
 }   
// Commit transaction
tm.commit(txnStatus);


println "---------------------------------------------------";
println "     " + scenarioDeletedCounter + " Scenarios deleted Sccessfully";
println "---------------------------------------------------";

} 
catch (Exception e)
{
  // Print Execption
  println "---------------------ERROR-------------------------";
  println(e);
  println "---------------------------------------------------";
  println "     FAILURE : Scenarios Deletion failed";
  println "---------------------------------------------------";
}

SHARE ON :


Related articles

March 25, 2024

Read in minutes

Getting started with the new Power BI Visual Calculations feature!

Power BI’s latest feature release, Visual Calculations, represents a paradigm shift in how users interact with data.      Rolled ...

February 20, 2024

Read in 5 minutes

Revolutionizing Data Engineering: The Power of Databricks’ Delta Live Tables and Unity Catalog

Databricks has emerged as a pivotal platform in the data engineering landscape, offering a comprehensive suite of tools designed to tackle the complexities of d...

November 20, 2023

Read in 5 minutes

AKABI’s Consultants Share Insights from Dataminds Connect 2023

Dataminds Connect 2023, a two-day event taking place in the charming city of Mechelen, Belgium, has proven to be a cornerstone in the world of IT and Microsoft ...


comments

Human and Machine Learning

March 17, 2019

Business Inteligence Event

Read in 2 minutes

I had the opportunity to attend the 2019 Gartner Data & Analytics Summit at London. Here is a wrap up of some notes I took during the sessions.

Few years ago, AI was a subject of fear for the future. Now it’s a fact, Machine Learning is part of the present. We are not anymore in a challenge Humans vs Machines, goal is to free human resources for higher end tasks. Humans and Machines…

You still have a problem with terms like Artificial Intelligence, Machine Learning? No worries, just replace them with “Augmented“.
Augmented Analytics, Augmented Data Management, Augmented Data Integration…

2019 will be Augmented. Not Human versus Machine but Human and Machine Learning at the service of a better Data World.

The new tools will let you operate as you used too but, in the background, will run Machine Learning algorithm to suggest you new vizualisations, unexpected facts, correlations, to save you from repetitive task…

  • All your integration flows have a common pattern, your augmented tool will detect it and propose you to create a new template automatically.
  • You select a set of analytics, your augmented tool will propose a cool vizualisation.
  • You want to prepare a dataset, your augmented analytics will automatically suggest formatting corrections, data mapping and learn from your choices.

If you plan to buy a new tool this year, be sure this is part of the roadmap.

Any other trends for 2019?
Many other trends were presented by Gartner, here are a couple of recurring ones during the sessions :

  • NLP. Natural Language Processing, new tools should be able to accept natural language as input (which allow vocal input from Alexa, Cortona…).
  • DataOps. No-one will deny Data is a subject where requirements evolve quickly. This is thus a choice area to apply agile development methods. DataOps is a specialized version of DevOps practices. This fits perfectly in an augmented world where most repetitive tasks should be automated.

On a non-technical side :

  • Data Literacy. Being a good technician is not enough if you work in the data world. You need to understand data, how they are and can be presented. Your ability to communicate around the data is as important as your ability to manage them. This is what include the data literacy skills. Some training exists on the web, a must for any consultant.

And many more you can find on Gartner web site or at future events.

Enjoy 2019 with machines.13 Rue de la Libération, 5969 Itzig, Luxembourg

SHARE ON :


Related articles

March 25, 2024

Read in minutes

Getting started with the new Power BI Visual Calculations feature!

Power BI’s latest feature release, Visual Calculations, represents a paradigm shift in how users interact with data.      Rolled ...

February 20, 2024

Read in 5 minutes

Revolutionizing Data Engineering: The Power of Databricks’ Delta Live Tables and Unity Catalog

Databricks has emerged as a pivotal platform in the data engineering landscape, offering a comprehensive suite of tools designed to tackle the complexities of d...

November 28, 2023

Read in 10 minutes

L’IA générative et les LLMs pour une information accessible et des processus optimisés

Le mois dernier, Medhi Famibelle, Pascal Nguyen et moi avons assisté dans les locaux du Wagon (entreprise proposant des formations dans la data) à trois talks...


comments