Sep 11 11:46:09.191242 ip-10-0-142-145 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 11 11:46:09.191253 ip-10-0-142-145 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 11 11:46:09.191263 ip-10-0-142-145 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 11 11:46:09.191463 ip-10-0-142-145 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 11 11:46:19.269280 ip-10-0-142-145 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 11 11:46:19.269300 ip-10-0-142-145 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 0c799d88b8b84b82a2d9548206b320a0 --
Sep 11 11:48:15.989730 ip-10-0-142-145 systemd[1]: Starting Kubernetes Kubelet...
Sep 11 11:48:16.352451 ip-10-0-142-145 kubenswrapper[2579]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 11:48:16.352451 ip-10-0-142-145 kubenswrapper[2579]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 11 11:48:16.352451 ip-10-0-142-145 kubenswrapper[2579]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 11:48:16.352451 ip-10-0-142-145 kubenswrapper[2579]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 11 11:48:16.352451 ip-10-0-142-145 kubenswrapper[2579]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 11:48:16.353929 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.353835    2579 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 11 11:48:16.357950 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.357932    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 11:48:16.357950 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.357950    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 11:48:16.358017 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.357955    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 11:48:16.358017 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.357959    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 11:48:16.358017 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.357962    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 11:48:16.358017 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.357965    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 11:48:16.358017 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.357967    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 11:48:16.358017 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.357971    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 11:48:16.358017 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.357974    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 11:48:16.358017 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.357977    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 11:48:16.358017 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.357979    2579 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 11:48:16.358017 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.357982    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 11:48:16.358017 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.357985    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 11:48:16.358017 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.357988    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 11:48:16.358017 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.357990    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 11:48:16.358017 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.357993    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 11:48:16.358017 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.357996    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 11:48:16.358017 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.357998    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 11:48:16.358017 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358008    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 11:48:16.358017 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358013    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 11:48:16.358017 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358016    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 11:48:16.358017 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358019    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 11:48:16.358524 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358022    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 11:48:16.358524 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358025    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 11:48:16.358524 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358028    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 11:48:16.358524 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358031    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 11:48:16.358524 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358034    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 11:48:16.358524 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358036    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 11:48:16.358524 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358039    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 11:48:16.358524 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358041    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 11:48:16.358524 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358044    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 11:48:16.358524 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358046    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 11:48:16.358524 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358049    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 11:48:16.358524 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358053    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 11:48:16.358524 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358057    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 11:48:16.358524 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358060    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 11:48:16.358524 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358063    2579 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 11:48:16.358524 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358066    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 11:48:16.358524 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358069    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 11:48:16.358524 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358071    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 11:48:16.358524 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358074    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 11:48:16.358973 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358076    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 11:48:16.358973 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358079    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 11:48:16.358973 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358081    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 11:48:16.358973 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358084    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 11:48:16.358973 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358086    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 11:48:16.358973 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358089    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 11:48:16.358973 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358091    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 11:48:16.358973 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358094    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 11:48:16.358973 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358097    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 11:48:16.358973 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358101    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 11:48:16.358973 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358104    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 11:48:16.358973 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358107    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 11:48:16.358973 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358110    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 11:48:16.358973 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358112    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 11:48:16.358973 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358116    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 11:48:16.358973 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358118    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 11:48:16.358973 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358121    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 11:48:16.358973 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358124    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 11:48:16.358973 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358126    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 11:48:16.359444 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358129    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 11:48:16.359444 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358131    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 11:48:16.359444 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358134    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 11:48:16.359444 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358136    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 11:48:16.359444 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358139    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 11:48:16.359444 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358141    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 11:48:16.359444 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358144    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 11:48:16.359444 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358146    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 11:48:16.359444 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358148    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 11:48:16.359444 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358151    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 11:48:16.359444 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358154    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 11:48:16.359444 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358157    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 11:48:16.359444 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358160    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 11:48:16.359444 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358162    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 11:48:16.359444 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358165    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 11:48:16.359444 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358168    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 11:48:16.359444 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358170    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 11:48:16.359444 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358173    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 11:48:16.359444 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358175    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 11:48:16.359444 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358178    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 11:48:16.359965 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358180    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 11:48:16.359965 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358183    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 11:48:16.359965 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358186    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 11:48:16.359965 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358188    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 11:48:16.359965 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358191    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 11:48:16.359965 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358194    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 11:48:16.359965 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358197    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 11:48:16.359965 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358200    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 11:48:16.359965 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358203    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 11:48:16.359965 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358206    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 11:48:16.359965 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358208    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 11:48:16.359965 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358211    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 11:48:16.359965 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358214    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 11:48:16.359965 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358217    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 11:48:16.359965 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358220    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 11:48:16.359965 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358222    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 11:48:16.359965 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358225    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 11:48:16.359965 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358228    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 11:48:16.359965 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358231    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 11:48:16.359965 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358977    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 11:48:16.360456 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358985    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 11:48:16.360456 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358988    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 11:48:16.360456 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358991    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 11:48:16.360456 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358994    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 11:48:16.360456 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.358998    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 11:48:16.360456 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359001    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 11:48:16.360456 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359003    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 11:48:16.360456 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359006    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 11:48:16.360456 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359009    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 11:48:16.360456 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359011    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 11:48:16.360456 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359014    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 11:48:16.360456 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359017    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 11:48:16.360456 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359021    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 11:48:16.360456 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359024    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 11:48:16.360456 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359027    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 11:48:16.360456 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359030    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 11:48:16.360456 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359033    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 11:48:16.360456 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359037    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 11:48:16.360456 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359040    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 11:48:16.360922 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359044    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 11:48:16.360922 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359046    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 11:48:16.360922 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359049    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 11:48:16.360922 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359052    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 11:48:16.360922 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359054    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 11:48:16.360922 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359057    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 11:48:16.360922 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359060    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 11:48:16.360922 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359062    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 11:48:16.360922 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359065    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 11:48:16.360922 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359067    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 11:48:16.360922 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359070    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 11:48:16.360922 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359074    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 11:48:16.360922 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359078    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 11:48:16.360922 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359080    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 11:48:16.360922 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359083    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 11:48:16.360922 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359085    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 11:48:16.360922 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359088    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 11:48:16.360922 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359091    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 11:48:16.360922 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359094    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 11:48:16.360922 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359096    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 11:48:16.361511 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359099    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 11:48:16.361511 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359102    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 11:48:16.361511 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359105    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 11:48:16.361511 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359107    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 11:48:16.361511 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359110    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 11:48:16.361511 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359113    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 11:48:16.361511 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359115    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 11:48:16.361511 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359118    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 11:48:16.361511 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359121    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 11:48:16.361511 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359124    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 11:48:16.361511 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359127    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 11:48:16.361511 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359130    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 11:48:16.361511 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359133    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 11:48:16.361511 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359135    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 11:48:16.361511 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359138    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 11:48:16.361511 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359141    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 11:48:16.361511 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359143    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 11:48:16.361511 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359146    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 11:48:16.361511 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359148    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 11:48:16.361511 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359151    2579 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 11:48:16.361999 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359154    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 11:48:16.361999 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359156    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 11:48:16.361999 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359159    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 11:48:16.361999 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359161    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 11:48:16.361999 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359164    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 11:48:16.361999 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359166    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 11:48:16.361999 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359169    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 11:48:16.361999 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359172    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 11:48:16.361999 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359175    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 11:48:16.361999 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359178    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 11:48:16.361999 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359181    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 11:48:16.361999 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359184    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 11:48:16.361999 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359187    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 11:48:16.361999 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359190    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 11:48:16.361999 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359192    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 11:48:16.361999 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359195    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 11:48:16.361999 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359198    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 11:48:16.361999 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359200    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 11:48:16.361999 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359203    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 11:48:16.361999 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359205    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 11:48:16.362513 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359208    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 11:48:16.362513 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359211    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 11:48:16.362513 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359213    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 11:48:16.362513 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359216    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 11:48:16.362513 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359218    2579 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 11:48:16.362513 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359221    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 11:48:16.362513 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359224    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 11:48:16.362513 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359226    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 11:48:16.362513 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359229    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 11:48:16.362513 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359233    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 11:48:16.362513 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359237    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 11:48:16.362513 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359240    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 11:48:16.362513 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359242    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 11:48:16.362513 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359245    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 11:48:16.362513 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359248    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 11:48:16.362513 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359250    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 11:48:16.362513 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359253    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 11:48:16.362513 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359256    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 11:48:16.362513 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359258    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 11:48:16.362970 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359645    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 11:48:16.362970 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359651    2579 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 11:48:16.362970 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359654    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 11:48:16.362970 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359657    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 11:48:16.362970 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359660    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 11:48:16.362970 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359663    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 11:48:16.362970 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359666    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 11:48:16.362970 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359668    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 11:48:16.362970 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359671    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 11:48:16.362970 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359674    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 11:48:16.362970 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359677    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 11:48:16.362970 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359680    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 11:48:16.362970 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359682    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 11:48:16.362970 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359685    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 11:48:16.362970 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359687    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 11:48:16.362970 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359690    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 11:48:16.362970 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359692    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 11:48:16.362970 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359695    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 11:48:16.362970 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359699    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 11:48:16.362970 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359703    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 11:48:16.363486 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359707    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 11:48:16.363486 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359710    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 11:48:16.363486 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359713    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 11:48:16.363486 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359716    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 11:48:16.363486 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359718    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 11:48:16.363486 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359721    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 11:48:16.363486 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359724    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 11:48:16.363486 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359727    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 11:48:16.363486 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359730    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 11:48:16.363486 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359732    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 11:48:16.363486 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359735    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 11:48:16.363486 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359737    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 11:48:16.363486 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359740    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 11:48:16.363486 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359742    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 11:48:16.363486 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359747    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 11:48:16.363486 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359750    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 11:48:16.363486 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359752    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 11:48:16.363486 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359756    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 11:48:16.363486 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359758    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 11:48:16.363932 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359761    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 11:48:16.363932 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359764    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 11:48:16.363932 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359767    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 11:48:16.363932 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359769    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 11:48:16.363932 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359772    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 11:48:16.363932 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359774    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 11:48:16.363932 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359778    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 11:48:16.363932 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359780    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 11:48:16.363932 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359783    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 11:48:16.363932 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359785    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 11:48:16.363932 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359788    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 11:48:16.363932 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359790    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 11:48:16.363932 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359793    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 11:48:16.363932 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359795    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 11:48:16.363932 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359798    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 11:48:16.363932 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359801    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 11:48:16.363932 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359803    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 11:48:16.363932 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359806    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 11:48:16.363932 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359810    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 11:48:16.364381 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359813    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 11:48:16.364381 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359816    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 11:48:16.364381 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359818    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 11:48:16.364381 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359821    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 11:48:16.364381 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359824    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 11:48:16.364381 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359826    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 11:48:16.364381 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359829    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 11:48:16.364381 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359832    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 11:48:16.364381 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359836    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 11:48:16.364381 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359839    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 11:48:16.364381 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359841    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 11:48:16.364381 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359844    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 11:48:16.364381 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359848    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 11:48:16.364381 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359851    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 11:48:16.364381 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359854    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 11:48:16.364381 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359856    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 11:48:16.364381 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359859    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 11:48:16.364381 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359862    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 11:48:16.364381 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359864    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 11:48:16.364381 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359867    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 11:48:16.364947 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359870    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 11:48:16.364947 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359872    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 11:48:16.364947 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359876    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 11:48:16.364947 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359879    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 11:48:16.364947 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359881    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 11:48:16.364947 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359884    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 11:48:16.364947 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359886    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 11:48:16.364947 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359889    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 11:48:16.364947 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359891    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 11:48:16.364947 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359894    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 11:48:16.364947 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359896    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 11:48:16.364947 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359899    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 11:48:16.364947 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359901    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 11:48:16.364947 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359904    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 11:48:16.364947 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359907    2579 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 11:48:16.364947 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359909    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 11:48:16.364947 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359912    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 11:48:16.364947 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359915    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 11:48:16.364947 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359918    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 11:48:16.364947 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359921    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 11:48:16.365448 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.359925    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 11:48:16.365448 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360000    2579 flags.go:64] FLAG: --address="0.0.0.0"
Sep 11 11:48:16.365448 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360034    2579 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 11 11:48:16.365448 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360041    2579 flags.go:64] FLAG: --anonymous-auth="true"
Sep 11 11:48:16.365448 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360049    2579 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 11 11:48:16.365448 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360054    2579 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 11 11:48:16.365448 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360057    2579 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 11 11:48:16.365448 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360062    2579 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 11 11:48:16.365448 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360066    2579 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 11 11:48:16.365448 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360071    2579 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 11 11:48:16.365448 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360074    2579 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 11 11:48:16.365448 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360078    2579 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 11 11:48:16.365448 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360081    2579 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 11 11:48:16.365448 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360084    2579 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 11 11:48:16.365448 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360088    2579 flags.go:64] FLAG: --cgroup-root=""
Sep 11 11:48:16.365448 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360091    2579 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 11 11:48:16.365448 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360094    2579 flags.go:64] FLAG: --client-ca-file=""
Sep 11 11:48:16.365448 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360096    2579 flags.go:64] FLAG: --cloud-provider="external"
Sep 11 11:48:16.365448 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360099    2579 flags.go:64] FLAG: --cluster-dns="[]"
Sep 11 11:48:16.365448 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360104    2579 flags.go:64] FLAG: --cluster-domain=""
Sep 11 11:48:16.365448 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360107    2579 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 11 11:48:16.365448 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360110    2579 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 11 11:48:16.365448 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360113    2579 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 11 11:48:16.365448 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360116    2579 flags.go:64] FLAG: --container-log-max-files="5"
Sep 11 11:48:16.366040 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360120    2579 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 11 11:48:16.366040 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360123    2579 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 11 11:48:16.366040 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360126    2579 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 11 11:48:16.366040 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360129    2579 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 11 11:48:16.366040 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360132    2579 flags.go:64] FLAG: --contention-profiling="false"
Sep 11 11:48:16.366040 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360134    2579 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 11 11:48:16.366040 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360137    2579 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 11 11:48:16.366040 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360140    2579 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 11 11:48:16.366040 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360144    2579 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 11 11:48:16.366040 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360148    2579 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 11 11:48:16.366040 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360151    2579 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 11 11:48:16.366040 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360154    2579 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 11 11:48:16.366040 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360158    2579 flags.go:64] FLAG: --enable-load-reader="false"
Sep 11 11:48:16.366040 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360161    2579 flags.go:64] FLAG: --enable-server="true"
Sep 11 11:48:16.366040 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360165    2579 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 11 11:48:16.366040 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360170    2579 flags.go:64] FLAG: --event-burst="100"
Sep 11 11:48:16.366040 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360173    2579 flags.go:64] FLAG: --event-qps="50"
Sep 11 11:48:16.366040 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360176    2579 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 11 11:48:16.366040 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360180    2579 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 11 11:48:16.366040 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360183    2579 flags.go:64] FLAG: --eviction-hard=""
Sep 11 11:48:16.366040 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360186    2579 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 11 11:48:16.366040 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360189    2579 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 11 11:48:16.366040 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360192    2579 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 11 11:48:16.366040 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360195    2579 flags.go:64] FLAG: --eviction-soft=""
Sep 11 11:48:16.366040 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360198    2579 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 11 11:48:16.366679 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360200    2579 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 11 11:48:16.366679 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360203    2579 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 11 11:48:16.366679 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360206    2579 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 11 11:48:16.366679 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360209    2579 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 11 11:48:16.366679 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360212    2579 flags.go:64] FLAG: --fail-swap-on="true"
Sep 11 11:48:16.366679 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360215    2579 flags.go:64] FLAG: --feature-gates=""
Sep 11 11:48:16.366679 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360218    2579 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 11 11:48:16.366679 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360221    2579 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 11 11:48:16.366679 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360224    2579 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 11 11:48:16.366679 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360227    2579 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 11 11:48:16.366679 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360230    2579 flags.go:64] FLAG: --healthz-port="10248"
Sep 11 11:48:16.366679 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360233    2579 flags.go:64] FLAG: --help="false"
Sep 11 11:48:16.366679 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360236    2579 flags.go:64] FLAG: --hostname-override="ip-10-0-142-145.ec2.internal"
Sep 11 11:48:16.366679 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360239    2579 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 11 11:48:16.366679 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360242    2579 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 11 11:48:16.366679 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360245    2579 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 11 11:48:16.366679 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360249    2579 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 11 11:48:16.366679 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360252    2579 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 11 11:48:16.366679 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360255    2579 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 11 11:48:16.366679 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360259    2579 flags.go:64] FLAG: --image-service-endpoint=""
Sep 11 11:48:16.366679 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360262    2579 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 11 11:48:16.366679 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360265    2579 flags.go:64] FLAG: --kube-api-burst="100"
Sep 11 11:48:16.366679 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360268    2579 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 11 11:48:16.367275 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360271    2579 flags.go:64] FLAG: --kube-api-qps="50"
Sep 11 11:48:16.367275 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360275    2579 flags.go:64] FLAG: --kube-reserved=""
Sep 11 11:48:16.367275 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360278    2579 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 11 11:48:16.367275 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360281    2579 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 11 11:48:16.367275 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360284    2579 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 11 11:48:16.367275 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360287    2579 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 11 11:48:16.367275 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360289    2579 flags.go:64] FLAG: --lock-file=""
Sep 11 11:48:16.367275 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360292    2579 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 11 11:48:16.367275 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360295    2579 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 11 11:48:16.367275 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360298    2579 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 11 11:48:16.367275 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360303    2579 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 11 11:48:16.367275 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360306    2579 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 11 11:48:16.367275 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360309    2579 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 11 11:48:16.367275 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360312    2579 flags.go:64] FLAG: --logging-format="text"
Sep 11 11:48:16.367275 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360315    2579 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 11 11:48:16.367275 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360318    2579 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 11 11:48:16.367275 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360321    2579 flags.go:64] FLAG: --manifest-url=""
Sep 11 11:48:16.367275 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360323    2579 flags.go:64] FLAG: --manifest-url-header=""
Sep 11 11:48:16.367275 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360328    2579 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 11 11:48:16.367275 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360331    2579 flags.go:64] FLAG: --max-open-files="1000000"
Sep 11 11:48:16.367275 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360335    2579 flags.go:64] FLAG: --max-pods="110"
Sep 11 11:48:16.367275 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360338    2579 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 11 11:48:16.367275 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360341    2579 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 11 11:48:16.367275 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360345    2579 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 11 11:48:16.367275 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360347    2579 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 11 11:48:16.367898 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360350    2579 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 11 11:48:16.367898 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360354    2579 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 11 11:48:16.367898 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360357    2579 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 11 11:48:16.367898 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360365    2579 flags.go:64] FLAG: --node-status-max-images="50"
Sep 11 11:48:16.367898 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360368    2579 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 11 11:48:16.367898 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360371    2579 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 11 11:48:16.367898 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360375    2579 flags.go:64] FLAG: --pod-cidr=""
Sep 11 11:48:16.367898 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360378    2579 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 11 11:48:16.367898 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360384    2579 flags.go:64] FLAG: --pod-manifest-path=""
Sep 11 11:48:16.367898 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360400    2579 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 11 11:48:16.367898 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360404    2579 flags.go:64] FLAG: --pods-per-core="0"
Sep 11 11:48:16.367898 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360407    2579 flags.go:64] FLAG: --port="10250"
Sep 11 11:48:16.367898 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360410    2579 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 11 11:48:16.367898 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360413    2579 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0e60aef4894e720a6"
Sep 11 11:48:16.367898 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360416    2579 flags.go:64] FLAG: --qos-reserved=""
Sep 11 11:48:16.367898 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360419    2579 flags.go:64] FLAG: --read-only-port="10255"
Sep 11 11:48:16.367898 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360422    2579 flags.go:64] FLAG: --register-node="true"
Sep 11 11:48:16.367898 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360425    2579 flags.go:64] FLAG: --register-with-taints=""
Sep 11 11:48:16.367898 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360429    2579 flags.go:64] FLAG: --registry-burst="10"
Sep 11 11:48:16.367898 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360431    2579 flags.go:64] FLAG: --registry-qps="5"
Sep 11 11:48:16.367898 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360434    2579 flags.go:64] FLAG: --reserved-cpus=""
Sep 11 11:48:16.367898 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360437    2579 flags.go:64] FLAG: --reserved-memory=""
Sep 11 11:48:16.367898 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360441    2579 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 11 11:48:16.367898 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360444    2579 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 11 11:48:16.367898 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360447    2579 flags.go:64] FLAG: --rotate-certificates="false"
Sep 11 11:48:16.368553 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360450    2579 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 11 11:48:16.368553 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360453    2579 flags.go:64] FLAG: --runonce="false"
Sep 11 11:48:16.368553 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360455    2579 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 11 11:48:16.368553 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360458    2579 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 11 11:48:16.368553 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360461    2579 flags.go:64] FLAG: --seccomp-default="false"
Sep 11 11:48:16.368553 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360464    2579 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 11 11:48:16.368553 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360467    2579 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 11 11:48:16.368553 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360471    2579 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 11 11:48:16.368553 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360474    2579 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 11 11:48:16.368553 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360477    2579 flags.go:64] FLAG: --storage-driver-password="root"
Sep 11 11:48:16.368553 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360480    2579 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 11 11:48:16.368553 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360483    2579 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 11 11:48:16.368553 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360486    2579 flags.go:64] FLAG: --storage-driver-user="root"
Sep 11 11:48:16.368553 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360489    2579 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 11 11:48:16.368553 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360492    2579 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 11 11:48:16.368553 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360495    2579 flags.go:64] FLAG: --system-cgroups=""
Sep 11 11:48:16.368553 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360498    2579 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 11 11:48:16.368553 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360503    2579 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 11 11:48:16.368553 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360506    2579 flags.go:64] FLAG: --tls-cert-file=""
Sep 11 11:48:16.368553 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360509    2579 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 11 11:48:16.368553 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360513    2579 flags.go:64] FLAG: --tls-min-version=""
Sep 11 11:48:16.368553 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360516    2579 flags.go:64] FLAG: --tls-private-key-file=""
Sep 11 11:48:16.368553 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360519    2579 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 11 11:48:16.368553 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360522    2579 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 11 11:48:16.368553 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360524    2579 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 11 11:48:16.369161 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360527    2579 flags.go:64] FLAG: --v="2"
Sep 11 11:48:16.369161 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360531    2579 flags.go:64] FLAG: --version="false"
Sep 11 11:48:16.369161 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360535    2579 flags.go:64] FLAG: --vmodule=""
Sep 11 11:48:16.369161 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360538    2579 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 11 11:48:16.369161 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.360541    2579 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 11 11:48:16.369161 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360686    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 11:48:16.369161 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360690    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 11:48:16.369161 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360694    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 11:48:16.369161 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360696    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 11:48:16.369161 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360699    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 11:48:16.369161 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360702    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 11:48:16.369161 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360705    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 11:48:16.369161 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360709    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 11:48:16.369161 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360711    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 11:48:16.369161 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360714    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 11:48:16.369161 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360716    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 11:48:16.369161 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360719    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 11:48:16.369161 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360723    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 11:48:16.369161 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360727    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 11:48:16.369161 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360731    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 11:48:16.369161 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360734    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 11:48:16.369688 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360737    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 11:48:16.369688 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360740    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 11:48:16.369688 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360744    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 11:48:16.369688 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360746    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 11:48:16.369688 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360749    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 11:48:16.369688 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360752    2579 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 11:48:16.369688 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360754    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 11:48:16.369688 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360757    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 11:48:16.369688 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360759    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 11:48:16.369688 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360762    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 11:48:16.369688 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360765    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 11:48:16.369688 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360767    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 11:48:16.369688 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360770    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 11:48:16.369688 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360772    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 11:48:16.369688 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360775    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 11:48:16.369688 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360777    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 11:48:16.369688 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360780    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 11:48:16.369688 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360783    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 11:48:16.369688 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360785    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 11:48:16.369688 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360788    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 11:48:16.370170 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360791    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 11:48:16.370170 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360793    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 11:48:16.370170 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360796    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 11:48:16.370170 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360799    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 11:48:16.370170 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360802    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 11:48:16.370170 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360805    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 11:48:16.370170 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360807    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 11:48:16.370170 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360810    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 11:48:16.370170 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360812    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 11:48:16.370170 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360815    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 11:48:16.370170 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360818    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 11:48:16.370170 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360821    2579 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 11:48:16.370170 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360823    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 11:48:16.370170 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360826    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 11:48:16.370170 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360830    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 11:48:16.370170 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360835    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 11:48:16.370170 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360838    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 11:48:16.370170 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360841    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 11:48:16.370170 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360843    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 11:48:16.370903 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360846    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 11:48:16.370903 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360848    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 11:48:16.370903 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360851    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 11:48:16.370903 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360853    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 11:48:16.370903 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360856    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 11:48:16.370903 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360859    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 11:48:16.370903 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360861    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 11:48:16.370903 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360864    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 11:48:16.370903 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360866    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 11:48:16.370903 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360869    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 11:48:16.370903 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360871    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 11:48:16.370903 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360874    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 11:48:16.370903 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360876    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 11:48:16.370903 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360879    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 11:48:16.370903 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360882    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 11:48:16.370903 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360884    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 11:48:16.370903 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360888    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 11:48:16.370903 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360890    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 11:48:16.370903 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360893    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 11:48:16.370903 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360896    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 11:48:16.371660 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360898    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 11:48:16.371660 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360901    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 11:48:16.371660 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360903    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 11:48:16.371660 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360906    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 11:48:16.371660 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360908    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 11:48:16.371660 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360911    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 11:48:16.371660 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360914    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 11:48:16.371660 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360916    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 11:48:16.371660 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360919    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 11:48:16.371660 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360922    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 11:48:16.371660 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360925    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 11:48:16.371660 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360928    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 11:48:16.371660 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360930    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 11:48:16.371660 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360933    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 11:48:16.371660 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360936    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 11:48:16.371660 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360938    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 11:48:16.371660 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360941    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 11:48:16.371660 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360943    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 11:48:16.371660 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360946    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 11:48:16.371660 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360948    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 11:48:16.372495 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360951    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 11:48:16.372495 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360953    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 11:48:16.372495 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360956    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 11:48:16.372495 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.360959    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 11:48:16.372495 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.361546    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 11:48:16.372495 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.368883    2579 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 11 11:48:16.372495 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.368900    2579 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 11 11:48:16.372495 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.369468    2579 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 11 11:48:16.372495 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.369475    2579 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 11 11:48:16.372495 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369532    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 11:48:16.372495 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369536    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 11:48:16.372495 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369540    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 11:48:16.372495 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369543    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 11:48:16.372495 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369547    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 11:48:16.373140 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369551    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 11:48:16.373140 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369554    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 11:48:16.373140 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369558    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 11:48:16.373140 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369560    2579 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 11:48:16.373140 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369564    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 11:48:16.373140 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369567    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 11:48:16.373140 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369569    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 11:48:16.373140 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369572    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 11:48:16.373140 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369575    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 11:48:16.373140 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369579    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 11:48:16.373140 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369582    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 11:48:16.373140 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369585    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 11:48:16.373140 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369587    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 11:48:16.373140 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369590    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 11:48:16.373140 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369593    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 11:48:16.373140 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369595    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 11:48:16.373140 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369600    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 11:48:16.373140 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369603    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 11:48:16.373140 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369606    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 11:48:16.373889 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369609    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 11:48:16.373889 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369613    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 11:48:16.373889 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369616    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 11:48:16.373889 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369618    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 11:48:16.373889 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369621    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 11:48:16.373889 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369625    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 11:48:16.373889 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369628    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 11:48:16.373889 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369630    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 11:48:16.373889 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369633    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 11:48:16.373889 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369636    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 11:48:16.373889 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369638    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 11:48:16.373889 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369641    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 11:48:16.373889 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369643    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 11:48:16.373889 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369646    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 11:48:16.373889 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369648    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 11:48:16.373889 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369651    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 11:48:16.373889 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369654    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 11:48:16.373889 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369657    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 11:48:16.373889 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369659    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 11:48:16.373889 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369662    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 11:48:16.374494 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369665    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 11:48:16.374494 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369667    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 11:48:16.374494 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369670    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 11:48:16.374494 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369673    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 11:48:16.374494 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369676    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 11:48:16.374494 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369678    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 11:48:16.374494 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369681    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 11:48:16.374494 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369683    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 11:48:16.374494 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369686    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 11:48:16.374494 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369689    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 11:48:16.374494 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369691    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 11:48:16.374494 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369694    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 11:48:16.374494 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369697    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 11:48:16.374494 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369700    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 11:48:16.374494 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369702    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 11:48:16.374494 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369705    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 11:48:16.374494 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369708    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 11:48:16.374494 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369712    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 11:48:16.374494 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369715    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 11:48:16.374494 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369718    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 11:48:16.375020 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369720    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 11:48:16.375020 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369723    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 11:48:16.375020 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369726    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 11:48:16.375020 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369728    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 11:48:16.375020 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369731    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 11:48:16.375020 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369734    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 11:48:16.375020 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369736    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 11:48:16.375020 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369739    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 11:48:16.375020 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369741    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 11:48:16.375020 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369744    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 11:48:16.375020 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369746    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 11:48:16.375020 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369749    2579 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 11:48:16.375020 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369751    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 11:48:16.375020 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369754    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 11:48:16.375020 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369756    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 11:48:16.375020 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369759    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 11:48:16.375020 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369761    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 11:48:16.375020 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369764    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 11:48:16.375020 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369766    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 11:48:16.375020 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369769    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 11:48:16.375588 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369772    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 11:48:16.375588 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369774    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 11:48:16.375588 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369777    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 11:48:16.375588 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369780    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 11:48:16.375588 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369782    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 11:48:16.375588 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369785    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 11:48:16.375588 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369788    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 11:48:16.375588 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369791    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 11:48:16.375588 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369794    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 11:48:16.375588 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369797    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 11:48:16.375588 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369799    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 11:48:16.375588 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369802    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 11:48:16.375588 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369805    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 11:48:16.375588 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369807    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 11:48:16.375588 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369810    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 11:48:16.375948 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.369815    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 11:48:16.375948 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369925    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 11:48:16.375948 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369931    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 11:48:16.375948 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369934    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 11:48:16.375948 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369937    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 11:48:16.375948 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369940    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 11:48:16.375948 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369942    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 11:48:16.375948 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369945    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 11:48:16.375948 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369948    2579 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 11:48:16.375948 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369950    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 11:48:16.375948 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369953    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 11:48:16.375948 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369955    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 11:48:16.375948 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369958    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 11:48:16.375948 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369961    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 11:48:16.375948 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369963    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 11:48:16.376346 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369966    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 11:48:16.376346 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369968    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 11:48:16.376346 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369971    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 11:48:16.376346 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369973    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 11:48:16.376346 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369976    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 11:48:16.376346 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369979    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 11:48:16.376346 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369981    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 11:48:16.376346 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369984    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 11:48:16.376346 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369987    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 11:48:16.376346 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369989    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 11:48:16.376346 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369992    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 11:48:16.376346 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369995    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 11:48:16.376346 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.369997    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 11:48:16.376346 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370000    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 11:48:16.376346 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370003    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 11:48:16.376346 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370006    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 11:48:16.376346 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370008    2579 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 11:48:16.376346 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370011    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 11:48:16.376346 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370013    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 11:48:16.376346 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370016    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 11:48:16.376926 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370018    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 11:48:16.376926 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370021    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 11:48:16.376926 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370023    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 11:48:16.376926 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370026    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 11:48:16.376926 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370028    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 11:48:16.376926 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370031    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 11:48:16.376926 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370034    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 11:48:16.376926 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370036    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 11:48:16.376926 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370039    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 11:48:16.376926 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370041    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 11:48:16.376926 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370044    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 11:48:16.376926 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370046    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 11:48:16.376926 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370049    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 11:48:16.376926 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370051    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 11:48:16.376926 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370055    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 11:48:16.376926 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370057    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 11:48:16.376926 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370060    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 11:48:16.376926 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370062    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 11:48:16.376926 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370065    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 11:48:16.377801 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370067    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 11:48:16.377801 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370069    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 11:48:16.377801 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370072    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 11:48:16.377801 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370076    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 11:48:16.377801 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370080    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 11:48:16.377801 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370084    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 11:48:16.377801 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370087    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 11:48:16.377801 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370090    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 11:48:16.377801 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370093    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 11:48:16.377801 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370097    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 11:48:16.377801 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370101    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 11:48:16.377801 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370104    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 11:48:16.377801 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370107    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 11:48:16.377801 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370110    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 11:48:16.377801 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370112    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 11:48:16.377801 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370115    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 11:48:16.377801 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370117    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 11:48:16.377801 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370120    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 11:48:16.377801 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370122    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 11:48:16.378601 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370125    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 11:48:16.378601 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370128    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 11:48:16.378601 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370130    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 11:48:16.378601 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370133    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 11:48:16.378601 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370135    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 11:48:16.378601 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370138    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 11:48:16.378601 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370140    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 11:48:16.378601 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370143    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 11:48:16.378601 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370146    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 11:48:16.378601 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370148    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 11:48:16.378601 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370151    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 11:48:16.378601 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370154    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 11:48:16.378601 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370156    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 11:48:16.378601 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370159    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 11:48:16.378601 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370161    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 11:48:16.378601 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370164    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 11:48:16.378601 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370167    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 11:48:16.378601 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370169    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 11:48:16.378601 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370172    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 11:48:16.378601 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370175    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 11:48:16.379359 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370178    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 11:48:16.379359 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370180    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 11:48:16.379359 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370183    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 11:48:16.379359 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370186    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 11:48:16.379359 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370189    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 11:48:16.379359 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370191    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 11:48:16.379359 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:16.370194    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 11:48:16.379359 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.370199    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 11:48:16.379359 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.370420    2579 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 11 11:48:16.379359 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.373811    2579 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 11 11:48:16.379359 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.374653    2579 server.go:1019] "Starting client certificate rotation"
Sep 11 11:48:16.379359 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.374758    2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 11 11:48:16.379359 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.374790    2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 11 11:48:16.379714 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.377402    2579 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 11 11:48:16.379714 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.379190    2579 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 11 11:48:16.388506 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.388490    2579 log.go:25] "Validated CRI v1 runtime API"
Sep 11 11:48:16.391446 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.391430    2579 log.go:25] "Validated CRI v1 image API"
Sep 11 11:48:16.393358 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.393333    2579 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 11 11:48:16.395510 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.395488    2579 fs.go:135] Filesystem UUIDs: map[269292b7-7883-4c6d-a6f8-a7efbe9a9de2:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 94a46211-6862-43dd-9f42-4798f3b16b57:/dev/nvme0n1p4]
Sep 11 11:48:16.395581 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.395509    2579 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 11 11:48:16.401023 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.400920    2579 manager.go:216] Machine: {Timestamp:2026-09-11 11:48:16.399780067 +0000 UTC m=+0.297221802 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098188 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec271f854b27aa653f00b72649e532ba SystemUUID:ec271f85-4b27-aa65-3f00-b72649e532ba BootID:0c799d88-b8b8-4b82-a2d9-548206b320a0 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:1b:62:36:52:69 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:1b:62:36:52:69 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:42:8b:ac:62:f0:de Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 11 11:48:16.401023 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.401021    2579 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 11 11:48:16.401163 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.401097    2579 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 11 11:48:16.403654 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.403631    2579 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 11 11:48:16.403822 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.403657    2579 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-145.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 11 11:48:16.403876 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.403854    2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 11 11:48:16.404724 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.404712    2579 topology_manager.go:138] "Creating topology manager with none policy"
Sep 11 11:48:16.404759 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.404726    2579 container_manager_linux.go:309] "Creating device plugin manager"
Sep 11 11:48:16.404759 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.404734    2579 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 11 11:48:16.405449 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.405438    2579 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 11 11:48:16.406617 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.406608    2579 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 11 11:48:16.409985 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.409973    2579 state_mem.go:36] "Initialized new in-memory state store"
Sep 11 11:48:16.410100 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.410091    2579 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 11 11:48:16.410130 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.410120    2579 kubelet.go:486] "Attempting to sync node with API server"
Sep 11 11:48:16.410159 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.410131    2579 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 11 11:48:16.410159 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.410145    2579 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 11 11:48:16.410159 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.410155    2579 kubelet.go:398] "Adding apiserver pod source"
Sep 11 11:48:16.410237 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.410164    2579 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 11 11:48:16.412196 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.411648    2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 11 11:48:16.412196 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.411681    2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 11 11:48:16.413731 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.413708    2579 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 11 11:48:16.416241 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.416227    2579 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 11 11:48:16.416285 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.416257    2579 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 11 11:48:16.417605 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.417587    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 11 11:48:16.417605 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.417603    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 11 11:48:16.417605 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.417609    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 11 11:48:16.417732 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.417615    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 11 11:48:16.417732 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.417623    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 11 11:48:16.417732 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.417629    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 11 11:48:16.417732 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.417642    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 11 11:48:16.417732 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.417647    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 11 11:48:16.417732 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.417654    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 11 11:48:16.417732 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.417659    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 11 11:48:16.417732 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.417668    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 11 11:48:16.417732 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.417673    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 11 11:48:16.418334 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.418325    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 11 11:48:16.418334 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.418333    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 11 11:48:16.421610 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.421587    2579 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-145.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 11 11:48:16.421707 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:16.421688    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-145.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 11 11:48:16.421831 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:16.421811    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 11 11:48:16.421956 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.421943    2579 server.go:1268] "Started kubelet"
Sep 11 11:48:16.422027 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.421998    2579 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 11 11:48:16.422089 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.422025    2579 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 11 11:48:16.422122 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.422108    2579 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 11 11:48:16.423320 ip-10-0-142-145 systemd[1]: Started Kubernetes Kubelet.
Sep 11 11:48:16.423576 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.423554    2579 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 11 11:48:16.424562 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.424539    2579 server.go:310] "Adding debug handlers to kubelet server"
Sep 11 11:48:16.426494 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:16.426348    2579 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 11 11:48:16.429125 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.429108    2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 11 11:48:16.429545 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.429528    2579 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 11 11:48:16.429660 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:16.429641    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-145.ec2.internal\" not found"
Sep 11 11:48:16.430167 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.430147    2579 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 11 11:48:16.430167 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.430146    2579 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 11 11:48:16.430300 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.430172    2579 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 11 11:48:16.430300 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.430255    2579 reconstruct.go:97] "Volume reconstruction finished"
Sep 11 11:48:16.430300 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.430265    2579 reconciler.go:29] "Reconciler: start to sync state"
Sep 11 11:48:16.430495 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.430475    2579 factory.go:55] Registering systemd factory
Sep 11 11:48:16.430495 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.430494    2579 factory.go:223] Registration of the systemd container factory successfully
Sep 11 11:48:16.432041 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.432023    2579 factory.go:153] Registering CRI-O factory
Sep 11 11:48:16.432130 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.432045    2579 factory.go:223] Registration of the crio container factory successfully
Sep 11 11:48:16.432130 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.432097    2579 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 11 11:48:16.432323 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.432131    2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-v57ld"
Sep 11 11:48:16.432323 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.432116    2579 factory.go:103] Registering Raw factory
Sep 11 11:48:16.432323 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.432164    2579 manager.go:1196] Started watching for new ooms in manager
Sep 11 11:48:16.432541 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.432527    2579 manager.go:349] Starting recovery of all containers
Sep 11 11:48:16.434234 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:16.432212    2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-145.ec2.internal.18d441cc012ef552  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-145.ec2.internal,UID:ip-10-0-142-145.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-145.ec2.internal,},FirstTimestamp:2026-09-11 11:48:16.421918034 +0000 UTC m=+0.319359769,LastTimestamp:2026-09-11 11:48:16.421918034 +0000 UTC m=+0.319359769,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-145.ec2.internal,}"
Sep 11 11:48:16.437109 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:16.437078    2579 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-145.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 11 11:48:16.437207 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:16.437108    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 11 11:48:16.442201 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.442178    2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-v57ld"
Sep 11 11:48:16.445522 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.445501    2579 manager.go:354] Recovery completed
Sep 11 11:48:16.449951 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.449821    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 11:48:16.451994 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.451973    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-145.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 11:48:16.452076 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.452011    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-145.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 11:48:16.452076 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.452023    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-145.ec2.internal" event="NodeHasSufficientPID"
Sep 11 11:48:16.452689 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.452672    2579 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 11 11:48:16.452689 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.452683    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 11:48:16.452826 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.452685    2579 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 11 11:48:16.452826 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.452726    2579 state_mem.go:36] "Initialized new in-memory state store"
Sep 11 11:48:16.453881 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.453867    2579 policy_none.go:49] "None policy: Start"
Sep 11 11:48:16.453970 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.453887    2579 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 11 11:48:16.453970 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.453899    2579 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 11 11:48:16.454058 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.454017    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-145.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 11:48:16.454058 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.454044    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-145.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 11:48:16.454136 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.454059    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-145.ec2.internal" event="NodeHasSufficientPID"
Sep 11 11:48:16.454179 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:16.454099    2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-145.ec2.internal.18d441cc02f9e038  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-145.ec2.internal,UID:ip-10-0-142-145.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-142-145.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-142-145.ec2.internal,},FirstTimestamp:2026-09-11 11:48:16.451993656 +0000 UTC m=+0.349435398,LastTimestamp:2026-09-11 11:48:16.451993656 +0000 UTC m=+0.349435398,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-145.ec2.internal,}"
Sep 11 11:48:16.455011 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.454998    2579 policy_none.go:47] "Start"
Sep 11 11:48:16.483165 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.483147    2579 manager.go:337] "Starting Device Plugin manager"
Sep 11 11:48:16.483273 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:16.483185    2579 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 11 11:48:16.483273 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.483196    2579 server.go:87] "Starting device plugin registration server"
Sep 11 11:48:16.483701 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.483684    2579 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 11 11:48:16.502401 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.483705    2579 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 11 11:48:16.502401 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.483795    2579 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 11 11:48:16.502401 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.483879    2579 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 11 11:48:16.502401 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.483888    2579 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 11 11:48:16.502401 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:16.484652    2579 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 11 11:48:16.502401 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:16.484693    2579 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-145.ec2.internal\" not found"
Sep 11 11:48:16.568091 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.568042    2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 11 11:48:16.569321 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.569297    2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 11 11:48:16.569456 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.569354    2579 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 11 11:48:16.569456 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.569378    2579 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 11 11:48:16.569456 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:16.569439    2579 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 11 11:48:16.571872 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.571843    2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 11:48:16.583995 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.583979    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 11:48:16.584925 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.584909    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-145.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 11:48:16.584981 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.584945    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-145.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 11:48:16.584981 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.584958    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-145.ec2.internal" event="NodeHasSufficientPID"
Sep 11 11:48:16.585095 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.584986    2579 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-145.ec2.internal"
Sep 11 11:48:16.593573 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.593557    2579 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-145.ec2.internal"
Sep 11 11:48:16.593646 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:16.593579    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-145.ec2.internal\": node \"ip-10-0-142-145.ec2.internal\" not found"
Sep 11 11:48:16.614247 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:16.614166    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-145.ec2.internal\" not found"
Sep 11 11:48:16.670195 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.670166    2579 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-142-145.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-145.ec2.internal"]
Sep 11 11:48:16.670338 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.670245    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 11:48:16.671841 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.671825    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-145.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 11:48:16.671902 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.671867    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-145.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 11:48:16.671902 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.671878    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-145.ec2.internal" event="NodeHasSufficientPID"
Sep 11 11:48:16.673266 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.673253    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 11:48:16.673387 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.673364    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-145.ec2.internal"
Sep 11 11:48:16.673447 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.673414    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 11:48:16.673989 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.673973    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-145.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 11:48:16.674050 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.674005    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-145.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 11:48:16.674050 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.674023    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-145.ec2.internal" event="NodeHasSufficientPID"
Sep 11 11:48:16.674114 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.673975    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-145.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 11:48:16.674114 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.674072    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-145.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 11:48:16.674114 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.674083    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-145.ec2.internal" event="NodeHasSufficientPID"
Sep 11 11:48:16.675265 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.675244    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-145.ec2.internal"
Sep 11 11:48:16.675265 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.675270    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 11:48:16.675949 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.675932    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-145.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 11:48:16.676011 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.675958    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-145.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 11:48:16.676011 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.675969    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-145.ec2.internal" event="NodeHasSufficientPID"
Sep 11 11:48:16.699099 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:16.699074    2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-145.ec2.internal\" not found" node="ip-10-0-142-145.ec2.internal"
Sep 11 11:48:16.702412 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:16.702381    2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-145.ec2.internal\" not found" node="ip-10-0-142-145.ec2.internal"
Sep 11 11:48:16.714329 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:16.714306    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-145.ec2.internal\" not found"
Sep 11 11:48:16.732482 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.732455    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0a88adf83eef5bb51fa329190cac6245-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-145.ec2.internal\" (UID: \"0a88adf83eef5bb51fa329190cac6245\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-145.ec2.internal"
Sep 11 11:48:16.732576 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.732488    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0a88adf83eef5bb51fa329190cac6245-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-145.ec2.internal\" (UID: \"0a88adf83eef5bb51fa329190cac6245\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-145.ec2.internal"
Sep 11 11:48:16.732576 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.732506    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3d9525f1a3218f8a340d89c0b17b6cdb-config\") pod \"kube-apiserver-proxy-ip-10-0-142-145.ec2.internal\" (UID: \"3d9525f1a3218f8a340d89c0b17b6cdb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-145.ec2.internal"
Sep 11 11:48:16.814676 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:16.814646    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-145.ec2.internal\" not found"
Sep 11 11:48:16.832977 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.832947    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0a88adf83eef5bb51fa329190cac6245-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-145.ec2.internal\" (UID: \"0a88adf83eef5bb51fa329190cac6245\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-145.ec2.internal"
Sep 11 11:48:16.833074 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.832982    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0a88adf83eef5bb51fa329190cac6245-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-145.ec2.internal\" (UID: \"0a88adf83eef5bb51fa329190cac6245\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-145.ec2.internal"
Sep 11 11:48:16.833074 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.832998    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3d9525f1a3218f8a340d89c0b17b6cdb-config\") pod \"kube-apiserver-proxy-ip-10-0-142-145.ec2.internal\" (UID: \"3d9525f1a3218f8a340d89c0b17b6cdb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-145.ec2.internal"
Sep 11 11:48:16.833074 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.833010    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0a88adf83eef5bb51fa329190cac6245-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-145.ec2.internal\" (UID: \"0a88adf83eef5bb51fa329190cac6245\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-145.ec2.internal"
Sep 11 11:48:16.833074 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.833036    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0a88adf83eef5bb51fa329190cac6245-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-145.ec2.internal\" (UID: \"0a88adf83eef5bb51fa329190cac6245\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-145.ec2.internal"
Sep 11 11:48:16.833074 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:16.833022    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3d9525f1a3218f8a340d89c0b17b6cdb-config\") pod \"kube-apiserver-proxy-ip-10-0-142-145.ec2.internal\" (UID: \"3d9525f1a3218f8a340d89c0b17b6cdb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-145.ec2.internal"
Sep 11 11:48:16.915378 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:16.915305    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-145.ec2.internal\" not found"
Sep 11 11:48:17.003079 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:17.003041    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-145.ec2.internal"
Sep 11 11:48:17.005713 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:17.005688    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-145.ec2.internal"
Sep 11 11:48:17.016463 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:17.016445    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-145.ec2.internal\" not found"
Sep 11 11:48:17.120776 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:17.120728    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-145.ec2.internal\" not found"
Sep 11 11:48:17.228101 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:17.228012    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-145.ec2.internal\" not found"
Sep 11 11:48:17.329520 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:17.329469    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-145.ec2.internal\" not found"
Sep 11 11:48:17.375589 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:17.375546    2579 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 11 11:48:17.376094 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:17.375724    2579 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 11 11:48:17.430527 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:17.430478    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-145.ec2.internal\" not found"
Sep 11 11:48:17.446069 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:17.446022    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-10 11:43:16 +0000 UTC" deadline="2028-06-17 08:49:06.995219532 +0000 UTC"
Sep 11 11:48:17.446069 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:17.446061    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15477h0m49.549163534s"
Sep 11 11:48:17.460136 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:17.460104    2579 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 11:48:17.497104 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:17.497034    2579 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 11:48:17.530848 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:17.530812    2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-145.ec2.internal"
Sep 11 11:48:17.545336 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:17.545318    2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 11 11:48:17.546188 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:17.546170    2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-145.ec2.internal"
Sep 11 11:48:17.563120 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:17.563094    2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 11 11:48:17.667699 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:17.667666    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d9525f1a3218f8a340d89c0b17b6cdb.slice/crio-e8a647904c1c642553160406855f287d192e8e28a327ce27be029ae8166067be WatchSource:0}: Error finding container e8a647904c1c642553160406855f287d192e8e28a327ce27be029ae8166067be: Status 404 returned error can't find the container with id e8a647904c1c642553160406855f287d192e8e28a327ce27be029ae8166067be
Sep 11 11:48:17.673970 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:17.673951    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 11:48:17.737273 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:17.737236    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a88adf83eef5bb51fa329190cac6245.slice/crio-e4cd1382466494bb5697c0281827d321bc8698c471c17fb0c719696feaa38689 WatchSource:0}: Error finding container e4cd1382466494bb5697c0281827d321bc8698c471c17fb0c719696feaa38689: Status 404 returned error can't find the container with id e4cd1382466494bb5697c0281827d321bc8698c471c17fb0c719696feaa38689
Sep 11 11:48:17.873000 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:17.872899    2579 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 11:48:18.412875 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.412829    2579 apiserver.go:52] "Watching apiserver"
Sep 11 11:48:18.421586 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.421551    2579 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 11 11:48:18.422835 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.422801    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-142-145.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kbzcr","openshift-cluster-node-tuning-operator/tuned-vrgj4","openshift-image-registry/node-ca-t7cd7","openshift-multus/multus-lh5dd","openshift-multus/network-metrics-daemon-krxsv","openshift-network-diagnostics/network-check-target-dqc42","openshift-ovn-kubernetes/ovnkube-node-2v285","kube-system/global-pull-secret-syncer-79ltx","kube-system/konnectivity-agent-dxf9l","openshift-dns/node-resolver-2jqbl","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-145.ec2.internal","openshift-multus/multus-additional-cni-plugins-4jthr","openshift-network-operator/iptables-alerter-pc2js"]
Sep 11 11:48:18.424507 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.424483    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dqc42"
Sep 11 11:48:18.424629 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:18.424572    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dqc42" podUID="711f1752-03c4-4f9a-bb1b-cb7bf8cac9cd"
Sep 11 11:48:18.425980 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.425947    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kbzcr"
Sep 11 11:48:18.427397 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.427376    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-vrgj4"
Sep 11 11:48:18.428722 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.428703    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-t7cd7"
Sep 11 11:48:18.429500 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.429454    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-bgmgc\""
Sep 11 11:48:18.429605 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.429548    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 11 11:48:18.429669 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.429454    2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 11 11:48:18.430229 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.429790    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 11 11:48:18.430229 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.430037    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 11 11:48:18.435731 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.433217    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-662nm\""
Sep 11 11:48:18.435731 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.433950    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 11 11:48:18.435731 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.434008    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 11 11:48:18.435731 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.434146    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 11 11:48:18.435731 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.434205    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-ljqjh\""
Sep 11 11:48:18.435731 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.434284    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 11 11:48:18.435731 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.434430    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 11 11:48:18.435731 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.434575    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-pc2js"
Sep 11 11:48:18.435731 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.434653    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-krxsv"
Sep 11 11:48:18.435731 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:18.434753    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-krxsv" podUID="fa7be6d8-e93e-49af-8782-14c8080fef33"
Sep 11 11:48:18.436653 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.436632    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4jthr"
Sep 11 11:48:18.437349 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.437287    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 11 11:48:18.437631 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.437610    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-xdxn9\""
Sep 11 11:48:18.437861 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.437838    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 11 11:48:18.438052 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.438036    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 11 11:48:18.438915 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.438468    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79ltx"
Sep 11 11:48:18.438915 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:18.438549    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79ltx" podUID="70994cc4-c221-4ab7-99ec-aed7c86955de"
Sep 11 11:48:18.439960 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.439788    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 11 11:48:18.441496 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.440150    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-dxf9l"
Sep 11 11:48:18.441496 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.440622    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 11 11:48:18.441496 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.440888    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-xrmth\""
Sep 11 11:48:18.441496 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.440939    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 11 11:48:18.441496 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.441348    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 11 11:48:18.444493 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.442288    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 11 11:48:18.444493 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.442369    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7872368e-69ce-4a5c-9773-e2d02b8f06ba-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4jthr\" (UID: \"7872368e-69ce-4a5c-9773-e2d02b8f06ba\") " pod="openshift-multus/multus-additional-cni-plugins-4jthr"
Sep 11 11:48:18.444493 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.442439    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9fbe0c28-1c37-4360-b183-dd216ce48906-etc-tuned\") pod \"tuned-vrgj4\" (UID: \"9fbe0c28-1c37-4360-b183-dd216ce48906\") " pod="openshift-cluster-node-tuning-operator/tuned-vrgj4"
Sep 11 11:48:18.444493 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.442519    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7s55\" (UniqueName: \"kubernetes.io/projected/5f1597c3-cf23-4523-bb1b-9003cc6f4b2a-kube-api-access-j7s55\") pod \"iptables-alerter-pc2js\" (UID: \"5f1597c3-cf23-4523-bb1b-9003cc6f4b2a\") " pod="openshift-network-operator/iptables-alerter-pc2js"
Sep 11 11:48:18.444493 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.442543    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxl2q\" (UniqueName: \"kubernetes.io/projected/fa7be6d8-e93e-49af-8782-14c8080fef33-kube-api-access-sxl2q\") pod \"network-metrics-daemon-krxsv\" (UID: \"fa7be6d8-e93e-49af-8782-14c8080fef33\") " pod="openshift-multus/network-metrics-daemon-krxsv"
Sep 11 11:48:18.444493 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.442585    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bcc619b6-bfb8-4cce-bf0c-b714377c7598-registration-dir\") pod \"aws-ebs-csi-driver-node-kbzcr\" (UID: \"bcc619b6-bfb8-4cce-bf0c-b714377c7598\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kbzcr"
Sep 11 11:48:18.444493 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.442608    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9fbe0c28-1c37-4360-b183-dd216ce48906-etc-sysconfig\") pod \"tuned-vrgj4\" (UID: \"9fbe0c28-1c37-4360-b183-dd216ce48906\") " pod="openshift-cluster-node-tuning-operator/tuned-vrgj4"
Sep 11 11:48:18.444493 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.442625    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9fbe0c28-1c37-4360-b183-dd216ce48906-sys\") pod \"tuned-vrgj4\" (UID: \"9fbe0c28-1c37-4360-b183-dd216ce48906\") " pod="openshift-cluster-node-tuning-operator/tuned-vrgj4"
Sep 11 11:48:18.444493 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.442651    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0f52f2ea-a695-422d-a16c-fa3db1f93041-host\") pod \"node-ca-t7cd7\" (UID: \"0f52f2ea-a695-422d-a16c-fa3db1f93041\") " pod="openshift-image-registry/node-ca-t7cd7"
Sep 11 11:48:18.444493 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.442666    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa7be6d8-e93e-49af-8782-14c8080fef33-metrics-certs\") pod \"network-metrics-daemon-krxsv\" (UID: \"fa7be6d8-e93e-49af-8782-14c8080fef33\") " pod="openshift-multus/network-metrics-daemon-krxsv"
Sep 11 11:48:18.444493 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.442685    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 11 11:48:18.444493 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.442681    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bcc619b6-bfb8-4cce-bf0c-b714377c7598-etc-selinux\") pod \"aws-ebs-csi-driver-node-kbzcr\" (UID: \"bcc619b6-bfb8-4cce-bf0c-b714377c7598\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kbzcr"
Sep 11 11:48:18.444493 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.442864    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7872368e-69ce-4a5c-9773-e2d02b8f06ba-system-cni-dir\") pod \"multus-additional-cni-plugins-4jthr\" (UID: \"7872368e-69ce-4a5c-9773-e2d02b8f06ba\") " pod="openshift-multus/multus-additional-cni-plugins-4jthr"
Sep 11 11:48:18.444493 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.442892    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7872368e-69ce-4a5c-9773-e2d02b8f06ba-cni-binary-copy\") pod \"multus-additional-cni-plugins-4jthr\" (UID: \"7872368e-69ce-4a5c-9773-e2d02b8f06ba\") " pod="openshift-multus/multus-additional-cni-plugins-4jthr"
Sep 11 11:48:18.444493 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.442925    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zp5xs\" (UniqueName: \"kubernetes.io/projected/7872368e-69ce-4a5c-9773-e2d02b8f06ba-kube-api-access-zp5xs\") pod \"multus-additional-cni-plugins-4jthr\" (UID: \"7872368e-69ce-4a5c-9773-e2d02b8f06ba\") " pod="openshift-multus/multus-additional-cni-plugins-4jthr"
Sep 11 11:48:18.444493 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.442958    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qbg9\" (UniqueName: \"kubernetes.io/projected/711f1752-03c4-4f9a-bb1b-cb7bf8cac9cd-kube-api-access-5qbg9\") pod \"network-check-target-dqc42\" (UID: \"711f1752-03c4-4f9a-bb1b-cb7bf8cac9cd\") " pod="openshift-network-diagnostics/network-check-target-dqc42"
Sep 11 11:48:18.444493 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.442982    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9fbe0c28-1c37-4360-b183-dd216ce48906-run\") pod \"tuned-vrgj4\" (UID: \"9fbe0c28-1c37-4360-b183-dd216ce48906\") " pod="openshift-cluster-node-tuning-operator/tuned-vrgj4"
Sep 11 11:48:18.445412 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.443023    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bcc619b6-bfb8-4cce-bf0c-b714377c7598-socket-dir\") pod \"aws-ebs-csi-driver-node-kbzcr\" (UID: \"bcc619b6-bfb8-4cce-bf0c-b714377c7598\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kbzcr"
Sep 11 11:48:18.445412 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.443051    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bcc619b6-bfb8-4cce-bf0c-b714377c7598-device-dir\") pod \"aws-ebs-csi-driver-node-kbzcr\" (UID: \"bcc619b6-bfb8-4cce-bf0c-b714377c7598\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kbzcr"
Sep 11 11:48:18.445412 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.443075    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7872368e-69ce-4a5c-9773-e2d02b8f06ba-cnibin\") pod \"multus-additional-cni-plugins-4jthr\" (UID: \"7872368e-69ce-4a5c-9773-e2d02b8f06ba\") " pod="openshift-multus/multus-additional-cni-plugins-4jthr"
Sep 11 11:48:18.445412 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.443107    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-xls4w\""
Sep 11 11:48:18.445412 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.443114    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9fbe0c28-1c37-4360-b183-dd216ce48906-etc-kubernetes\") pod \"tuned-vrgj4\" (UID: \"9fbe0c28-1c37-4360-b183-dd216ce48906\") " pod="openshift-cluster-node-tuning-operator/tuned-vrgj4"
Sep 11 11:48:18.445412 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.443141    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9fbe0c28-1c37-4360-b183-dd216ce48906-etc-systemd\") pod \"tuned-vrgj4\" (UID: \"9fbe0c28-1c37-4360-b183-dd216ce48906\") " pod="openshift-cluster-node-tuning-operator/tuned-vrgj4"
Sep 11 11:48:18.445412 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.443168    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9fbe0c28-1c37-4360-b183-dd216ce48906-tmp\") pod \"tuned-vrgj4\" (UID: \"9fbe0c28-1c37-4360-b183-dd216ce48906\") " pod="openshift-cluster-node-tuning-operator/tuned-vrgj4"
Sep 11 11:48:18.445412 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.443195    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5f1597c3-cf23-4523-bb1b-9003cc6f4b2a-host-slash\") pod \"iptables-alerter-pc2js\" (UID: \"5f1597c3-cf23-4523-bb1b-9003cc6f4b2a\") " pod="openshift-network-operator/iptables-alerter-pc2js"
Sep 11 11:48:18.445412 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.443219    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bcc619b6-bfb8-4cce-bf0c-b714377c7598-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kbzcr\" (UID: \"bcc619b6-bfb8-4cce-bf0c-b714377c7598\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kbzcr"
Sep 11 11:48:18.445412 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.443243    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9fbe0c28-1c37-4360-b183-dd216ce48906-etc-modprobe-d\") pod \"tuned-vrgj4\" (UID: \"9fbe0c28-1c37-4360-b183-dd216ce48906\") " pod="openshift-cluster-node-tuning-operator/tuned-vrgj4"
Sep 11 11:48:18.445412 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.443267    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9fbe0c28-1c37-4360-b183-dd216ce48906-etc-sysctl-conf\") pod \"tuned-vrgj4\" (UID: \"9fbe0c28-1c37-4360-b183-dd216ce48906\") " pod="openshift-cluster-node-tuning-operator/tuned-vrgj4"
Sep 11 11:48:18.445412 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.443290    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9fbe0c28-1c37-4360-b183-dd216ce48906-lib-modules\") pod \"tuned-vrgj4\" (UID: \"9fbe0c28-1c37-4360-b183-dd216ce48906\") " pod="openshift-cluster-node-tuning-operator/tuned-vrgj4"
Sep 11 11:48:18.445412 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.443357    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9fbe0c28-1c37-4360-b183-dd216ce48906-var-lib-kubelet\") pod \"tuned-vrgj4\" (UID: \"9fbe0c28-1c37-4360-b183-dd216ce48906\") " pod="openshift-cluster-node-tuning-operator/tuned-vrgj4"
Sep 11 11:48:18.445412 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.443415    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvf85\" (UniqueName: \"kubernetes.io/projected/9fbe0c28-1c37-4360-b183-dd216ce48906-kube-api-access-xvf85\") pod \"tuned-vrgj4\" (UID: \"9fbe0c28-1c37-4360-b183-dd216ce48906\") " pod="openshift-cluster-node-tuning-operator/tuned-vrgj4"
Sep 11 11:48:18.445412 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.443444    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-td5q7\" (UniqueName: \"kubernetes.io/projected/0f52f2ea-a695-422d-a16c-fa3db1f93041-kube-api-access-td5q7\") pod \"node-ca-t7cd7\" (UID: \"0f52f2ea-a695-422d-a16c-fa3db1f93041\") " pod="openshift-image-registry/node-ca-t7cd7"
Sep 11 11:48:18.445412 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.443486    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5f1597c3-cf23-4523-bb1b-9003cc6f4b2a-iptables-alerter-script\") pod \"iptables-alerter-pc2js\" (UID: \"5f1597c3-cf23-4523-bb1b-9003cc6f4b2a\") " pod="openshift-network-operator/iptables-alerter-pc2js"
Sep 11 11:48:18.445412 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.443519    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75hnp\" (UniqueName: \"kubernetes.io/projected/bcc619b6-bfb8-4cce-bf0c-b714377c7598-kube-api-access-75hnp\") pod \"aws-ebs-csi-driver-node-kbzcr\" (UID: \"bcc619b6-bfb8-4cce-bf0c-b714377c7598\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kbzcr"
Sep 11 11:48:18.446302 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.443544    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9fbe0c28-1c37-4360-b183-dd216ce48906-etc-sysctl-d\") pod \"tuned-vrgj4\" (UID: \"9fbe0c28-1c37-4360-b183-dd216ce48906\") " pod="openshift-cluster-node-tuning-operator/tuned-vrgj4"
Sep 11 11:48:18.446302 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.443567    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9fbe0c28-1c37-4360-b183-dd216ce48906-host\") pod \"tuned-vrgj4\" (UID: \"9fbe0c28-1c37-4360-b183-dd216ce48906\") " pod="openshift-cluster-node-tuning-operator/tuned-vrgj4"
Sep 11 11:48:18.446302 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.443592    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0f52f2ea-a695-422d-a16c-fa3db1f93041-serviceca\") pod \"node-ca-t7cd7\" (UID: \"0f52f2ea-a695-422d-a16c-fa3db1f93041\") " pod="openshift-image-registry/node-ca-t7cd7"
Sep 11 11:48:18.446302 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.443620    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bcc619b6-bfb8-4cce-bf0c-b714377c7598-sys-fs\") pod \"aws-ebs-csi-driver-node-kbzcr\" (UID: \"bcc619b6-bfb8-4cce-bf0c-b714377c7598\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kbzcr"
Sep 11 11:48:18.446302 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.443644    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7872368e-69ce-4a5c-9773-e2d02b8f06ba-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4jthr\" (UID: \"7872368e-69ce-4a5c-9773-e2d02b8f06ba\") " pod="openshift-multus/multus-additional-cni-plugins-4jthr"
Sep 11 11:48:18.446302 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.443670    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7872368e-69ce-4a5c-9773-e2d02b8f06ba-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4jthr\" (UID: \"7872368e-69ce-4a5c-9773-e2d02b8f06ba\") " pod="openshift-multus/multus-additional-cni-plugins-4jthr"
Sep 11 11:48:18.446302 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.443938    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-2jqbl"
Sep 11 11:48:18.446302 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.444061    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2v285"
Sep 11 11:48:18.446302 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.444734    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lh5dd"
Sep 11 11:48:18.446302 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.445116    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 11 11:48:18.446302 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.445133    2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 11 11:48:18.447176 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.446889    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 11 11:48:18.447176 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.447024    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 11 11:48:18.447436 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.447403    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 11 11:48:18.447542 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.447524    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-qk8tc\""
Sep 11 11:48:18.447738 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.447700    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 11 11:48:18.448298 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.448275    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 11 11:48:18.448472 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.448450    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 11 11:48:18.449182 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.449159    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 11 11:48:18.449587 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.449566    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-hx66l\""
Sep 11 11:48:18.450985 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.450847    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 11 11:48:18.450985 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.450880    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-g6p6d\""
Sep 11 11:48:18.451130 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.451050    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 11 11:48:18.473888 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.473845    2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-kdsfx"
Sep 11 11:48:18.483085 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.482756    2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-kdsfx"
Sep 11 11:48:18.531195 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.531165    2579 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 11 11:48:18.544813 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.544704    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7872368e-69ce-4a5c-9773-e2d02b8f06ba-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4jthr\" (UID: \"7872368e-69ce-4a5c-9773-e2d02b8f06ba\") " pod="openshift-multus/multus-additional-cni-plugins-4jthr"
Sep 11 11:48:18.544813 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.544755    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2dd13752-864e-44e6-a1a5-986e623fbd7a-host-run-k8s-cni-cncf-io\") pod \"multus-lh5dd\" (UID: \"2dd13752-864e-44e6-a1a5-986e623fbd7a\") " pod="openshift-multus/multus-lh5dd"
Sep 11 11:48:18.544813 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.544787    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bcc619b6-bfb8-4cce-bf0c-b714377c7598-registration-dir\") pod \"aws-ebs-csi-driver-node-kbzcr\" (UID: \"bcc619b6-bfb8-4cce-bf0c-b714377c7598\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kbzcr"
Sep 11 11:48:18.544813 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.544814    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cdaded97-b0ac-4688-8721-813fc905c103-tmp-dir\") pod \"node-resolver-2jqbl\" (UID: \"cdaded97-b0ac-4688-8721-813fc905c103\") " pod="openshift-dns/node-resolver-2jqbl"
Sep 11 11:48:18.545167 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.544838    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8fc69afd-c7cf-442c-8692-3ec71caffa8b-host-slash\") pod \"ovnkube-node-2v285\" (UID: \"8fc69afd-c7cf-442c-8692-3ec71caffa8b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2v285"
Sep 11 11:48:18.545167 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.544862    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2dd13752-864e-44e6-a1a5-986e623fbd7a-multus-socket-dir-parent\") pod \"multus-lh5dd\" (UID: \"2dd13752-864e-44e6-a1a5-986e623fbd7a\") " pod="openshift-multus/multus-lh5dd"
Sep 11 11:48:18.545167 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.544882    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2dd13752-864e-44e6-a1a5-986e623fbd7a-host-var-lib-cni-bin\") pod \"multus-lh5dd\" (UID: \"2dd13752-864e-44e6-a1a5-986e623fbd7a\") " pod="openshift-multus/multus-lh5dd"
Sep 11 11:48:18.545167 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.544902    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2dd13752-864e-44e6-a1a5-986e623fbd7a-hostroot\") pod \"multus-lh5dd\" (UID: \"2dd13752-864e-44e6-a1a5-986e623fbd7a\") " pod="openshift-multus/multus-lh5dd"
Sep 11 11:48:18.545167 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.544923    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2dd13752-864e-44e6-a1a5-986e623fbd7a-host-run-multus-certs\") pod \"multus-lh5dd\" (UID: \"2dd13752-864e-44e6-a1a5-986e623fbd7a\") " pod="openshift-multus/multus-lh5dd"
Sep 11 11:48:18.545167 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.544921    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bcc619b6-bfb8-4cce-bf0c-b714377c7598-registration-dir\") pod \"aws-ebs-csi-driver-node-kbzcr\" (UID: \"bcc619b6-bfb8-4cce-bf0c-b714377c7598\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kbzcr"
Sep 11 11:48:18.545167 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.544945    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2dd13752-864e-44e6-a1a5-986e623fbd7a-host-var-lib-cni-multus\") pod \"multus-lh5dd\" (UID: \"2dd13752-864e-44e6-a1a5-986e623fbd7a\") " pod="openshift-multus/multus-lh5dd"
Sep 11 11:48:18.545167 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.544968    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2dd13752-864e-44e6-a1a5-986e623fbd7a-etc-kubernetes\") pod \"multus-lh5dd\" (UID: \"2dd13752-864e-44e6-a1a5-986e623fbd7a\") " pod="openshift-multus/multus-lh5dd"
Sep 11 11:48:18.545167 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.545005    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9fbe0c28-1c37-4360-b183-dd216ce48906-sys\") pod \"tuned-vrgj4\" (UID: \"9fbe0c28-1c37-4360-b183-dd216ce48906\") " pod="openshift-cluster-node-tuning-operator/tuned-vrgj4"
Sep 11 11:48:18.545167 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.545038    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bcc619b6-bfb8-4cce-bf0c-b714377c7598-etc-selinux\") pod \"aws-ebs-csi-driver-node-kbzcr\" (UID: \"bcc619b6-bfb8-4cce-bf0c-b714377c7598\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kbzcr"
Sep 11 11:48:18.545167 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.545065    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7872368e-69ce-4a5c-9773-e2d02b8f06ba-system-cni-dir\") pod \"multus-additional-cni-plugins-4jthr\" (UID: \"7872368e-69ce-4a5c-9773-e2d02b8f06ba\") " pod="openshift-multus/multus-additional-cni-plugins-4jthr"
Sep 11 11:48:18.545167 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.545094    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9fbe0c28-1c37-4360-b183-dd216ce48906-sys\") pod \"tuned-vrgj4\" (UID: \"9fbe0c28-1c37-4360-b183-dd216ce48906\") " pod="openshift-cluster-node-tuning-operator/tuned-vrgj4"
Sep 11 11:48:18.545167 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.545106    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zp5xs\" (UniqueName: \"kubernetes.io/projected/7872368e-69ce-4a5c-9773-e2d02b8f06ba-kube-api-access-zp5xs\") pod \"multus-additional-cni-plugins-4jthr\" (UID: \"7872368e-69ce-4a5c-9773-e2d02b8f06ba\") " pod="openshift-multus/multus-additional-cni-plugins-4jthr"
Sep 11 11:48:18.545842 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.545230    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bcc619b6-bfb8-4cce-bf0c-b714377c7598-etc-selinux\") pod \"aws-ebs-csi-driver-node-kbzcr\" (UID: \"bcc619b6-bfb8-4cce-bf0c-b714377c7598\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kbzcr"
Sep 11 11:48:18.545842 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.545273    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/70994cc4-c221-4ab7-99ec-aed7c86955de-dbus\") pod \"global-pull-secret-syncer-79ltx\" (UID: \"70994cc4-c221-4ab7-99ec-aed7c86955de\") " pod="kube-system/global-pull-secret-syncer-79ltx"
Sep 11 11:48:18.545842 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.545309    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8fc69afd-c7cf-442c-8692-3ec71caffa8b-ovnkube-config\") pod \"ovnkube-node-2v285\" (UID: \"8fc69afd-c7cf-442c-8692-3ec71caffa8b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2v285"
Sep 11 11:48:18.545842 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.545337    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zp4rk\" (UniqueName: \"kubernetes.io/projected/2dd13752-864e-44e6-a1a5-986e623fbd7a-kube-api-access-zp4rk\") pod \"multus-lh5dd\" (UID: \"2dd13752-864e-44e6-a1a5-986e623fbd7a\") " pod="openshift-multus/multus-lh5dd"
Sep 11 11:48:18.545842 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.545371    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/70994cc4-c221-4ab7-99ec-aed7c86955de-kubelet-config\") pod \"global-pull-secret-syncer-79ltx\" (UID: \"70994cc4-c221-4ab7-99ec-aed7c86955de\") " pod="kube-system/global-pull-secret-syncer-79ltx"
Sep 11 11:48:18.545842 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.545397    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/70994cc4-c221-4ab7-99ec-aed7c86955de-original-pull-secret\") pod \"global-pull-secret-syncer-79ltx\" (UID: \"70994cc4-c221-4ab7-99ec-aed7c86955de\") " pod="kube-system/global-pull-secret-syncer-79ltx"
Sep 11 11:48:18.545842 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.545423    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8fc69afd-c7cf-442c-8692-3ec71caffa8b-run-systemd\") pod \"ovnkube-node-2v285\" (UID: \"8fc69afd-c7cf-442c-8692-3ec71caffa8b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2v285"
Sep 11 11:48:18.545842 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.545425    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7872368e-69ce-4a5c-9773-e2d02b8f06ba-system-cni-dir\") pod \"multus-additional-cni-plugins-4jthr\" (UID: \"7872368e-69ce-4a5c-9773-e2d02b8f06ba\") " pod="openshift-multus/multus-additional-cni-plugins-4jthr"
Sep 11 11:48:18.545842 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.545481    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qbg9\" (UniqueName: \"kubernetes.io/projected/711f1752-03c4-4f9a-bb1b-cb7bf8cac9cd-kube-api-access-5qbg9\") pod \"network-check-target-dqc42\" (UID: \"711f1752-03c4-4f9a-bb1b-cb7bf8cac9cd\") " pod="openshift-network-diagnostics/network-check-target-dqc42"
Sep 11 11:48:18.545842 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.545494    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7872368e-69ce-4a5c-9773-e2d02b8f06ba-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4jthr\" (UID: \"7872368e-69ce-4a5c-9773-e2d02b8f06ba\") " pod="openshift-multus/multus-additional-cni-plugins-4jthr"
Sep 11 11:48:18.545842 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.545510    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9fbe0c28-1c37-4360-b183-dd216ce48906-run\") pod \"tuned-vrgj4\" (UID: \"9fbe0c28-1c37-4360-b183-dd216ce48906\") " pod="openshift-cluster-node-tuning-operator/tuned-vrgj4"
Sep 11 11:48:18.545842 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.545647    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bcc619b6-bfb8-4cce-bf0c-b714377c7598-socket-dir\") pod \"aws-ebs-csi-driver-node-kbzcr\" (UID: \"bcc619b6-bfb8-4cce-bf0c-b714377c7598\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kbzcr"
Sep 11 11:48:18.545842 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.545686    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8fc69afd-c7cf-442c-8692-3ec71caffa8b-systemd-units\") pod \"ovnkube-node-2v285\" (UID: \"8fc69afd-c7cf-442c-8692-3ec71caffa8b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2v285"
Sep 11 11:48:18.545842 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.545735    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8fc69afd-c7cf-442c-8692-3ec71caffa8b-ovnkube-script-lib\") pod \"ovnkube-node-2v285\" (UID: \"8fc69afd-c7cf-442c-8692-3ec71caffa8b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2v285"
Sep 11 11:48:18.545842 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.545752    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9fbe0c28-1c37-4360-b183-dd216ce48906-run\") pod \"tuned-vrgj4\" (UID: \"9fbe0c28-1c37-4360-b183-dd216ce48906\") " pod="openshift-cluster-node-tuning-operator/tuned-vrgj4"
Sep 11 11:48:18.545842 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.545764    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8fc69afd-c7cf-442c-8692-3ec71caffa8b-run-openvswitch\") pod \"ovnkube-node-2v285\" (UID: \"8fc69afd-c7cf-442c-8692-3ec71caffa8b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2v285"
Sep 11 11:48:18.545842 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.545805    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9fbe0c28-1c37-4360-b183-dd216ce48906-etc-kubernetes\") pod \"tuned-vrgj4\" (UID: \"9fbe0c28-1c37-4360-b183-dd216ce48906\") " pod="openshift-cluster-node-tuning-operator/tuned-vrgj4"
Sep 11 11:48:18.546784 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.545836    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9fbe0c28-1c37-4360-b183-dd216ce48906-tmp\") pod \"tuned-vrgj4\" (UID: \"9fbe0c28-1c37-4360-b183-dd216ce48906\") " pod="openshift-cluster-node-tuning-operator/tuned-vrgj4"
Sep 11 11:48:18.546784 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.545864    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bcc619b6-bfb8-4cce-bf0c-b714377c7598-socket-dir\") pod \"aws-ebs-csi-driver-node-kbzcr\" (UID: \"bcc619b6-bfb8-4cce-bf0c-b714377c7598\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kbzcr"
Sep 11 11:48:18.546784 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.545875    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bcc619b6-bfb8-4cce-bf0c-b714377c7598-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kbzcr\" (UID: \"bcc619b6-bfb8-4cce-bf0c-b714377c7598\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kbzcr"
Sep 11 11:48:18.546784 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.545936    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8fc69afd-c7cf-442c-8692-3ec71caffa8b-env-overrides\") pod \"ovnkube-node-2v285\" (UID: \"8fc69afd-c7cf-442c-8692-3ec71caffa8b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2v285"
Sep 11 11:48:18.546784 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.545962    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cdaded97-b0ac-4688-8721-813fc905c103-hosts-file\") pod \"node-resolver-2jqbl\" (UID: \"cdaded97-b0ac-4688-8721-813fc905c103\") " pod="openshift-dns/node-resolver-2jqbl"
Sep 11 11:48:18.546784 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.545976    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9fbe0c28-1c37-4360-b183-dd216ce48906-etc-kubernetes\") pod \"tuned-vrgj4\" (UID: \"9fbe0c28-1c37-4360-b183-dd216ce48906\") " pod="openshift-cluster-node-tuning-operator/tuned-vrgj4"
Sep 11 11:48:18.546784 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.545988    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8fc69afd-c7cf-442c-8692-3ec71caffa8b-etc-openvswitch\") pod \"ovnkube-node-2v285\" (UID: \"8fc69afd-c7cf-442c-8692-3ec71caffa8b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2v285"
Sep 11 11:48:18.546784 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.546024    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9fbe0c28-1c37-4360-b183-dd216ce48906-etc-sysctl-conf\") pod \"tuned-vrgj4\" (UID: \"9fbe0c28-1c37-4360-b183-dd216ce48906\") " pod="openshift-cluster-node-tuning-operator/tuned-vrgj4"
Sep 11 11:48:18.546784 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.546053    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-td5q7\" (UniqueName: \"kubernetes.io/projected/0f52f2ea-a695-422d-a16c-fa3db1f93041-kube-api-access-td5q7\") pod \"node-ca-t7cd7\" (UID: \"0f52f2ea-a695-422d-a16c-fa3db1f93041\") " pod="openshift-image-registry/node-ca-t7cd7"
Sep 11 11:48:18.546784 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.546077    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75hnp\" (UniqueName: \"kubernetes.io/projected/bcc619b6-bfb8-4cce-bf0c-b714377c7598-kube-api-access-75hnp\") pod \"aws-ebs-csi-driver-node-kbzcr\" (UID: \"bcc619b6-bfb8-4cce-bf0c-b714377c7598\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kbzcr"
Sep 11 11:48:18.546784 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.546103    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7872368e-69ce-4a5c-9773-e2d02b8f06ba-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4jthr\" (UID: \"7872368e-69ce-4a5c-9773-e2d02b8f06ba\") " pod="openshift-multus/multus-additional-cni-plugins-4jthr"
Sep 11 11:48:18.546784 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.546167    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/43cde4a1-c7f9-443d-9f2f-0ab809b5d25a-agent-certs\") pod \"konnectivity-agent-dxf9l\" (UID: \"43cde4a1-c7f9-443d-9f2f-0ab809b5d25a\") " pod="kube-system/konnectivity-agent-dxf9l"
Sep 11 11:48:18.546784 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.546190    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8fc69afd-c7cf-442c-8692-3ec71caffa8b-log-socket\") pod \"ovnkube-node-2v285\" (UID: \"8fc69afd-c7cf-442c-8692-3ec71caffa8b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2v285"
Sep 11 11:48:18.546784 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.546211    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/43cde4a1-c7f9-443d-9f2f-0ab809b5d25a-konnectivity-ca\") pod \"konnectivity-agent-dxf9l\" (UID: \"43cde4a1-c7f9-443d-9f2f-0ab809b5d25a\") " pod="kube-system/konnectivity-agent-dxf9l"
Sep 11 11:48:18.546784 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.546233    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9fbe0c28-1c37-4360-b183-dd216ce48906-host\") pod \"tuned-vrgj4\" (UID: \"9fbe0c28-1c37-4360-b183-dd216ce48906\") " pod="openshift-cluster-node-tuning-operator/tuned-vrgj4"
Sep 11 11:48:18.546784 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.546254    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0f52f2ea-a695-422d-a16c-fa3db1f93041-serviceca\") pod \"node-ca-t7cd7\" (UID: \"0f52f2ea-a695-422d-a16c-fa3db1f93041\") " pod="openshift-image-registry/node-ca-t7cd7"
Sep 11 11:48:18.546784 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.546275    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2dd13752-864e-44e6-a1a5-986e623fbd7a-system-cni-dir\") pod \"multus-lh5dd\" (UID: \"2dd13752-864e-44e6-a1a5-986e623fbd7a\") " pod="openshift-multus/multus-lh5dd"
Sep 11 11:48:18.547710 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.546296    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2dd13752-864e-44e6-a1a5-986e623fbd7a-multus-conf-dir\") pod \"multus-lh5dd\" (UID: \"2dd13752-864e-44e6-a1a5-986e623fbd7a\") " pod="openshift-multus/multus-lh5dd"
Sep 11 11:48:18.547710 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.546317    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9fbe0c28-1c37-4360-b183-dd216ce48906-etc-tuned\") pod \"tuned-vrgj4\" (UID: \"9fbe0c28-1c37-4360-b183-dd216ce48906\") " pod="openshift-cluster-node-tuning-operator/tuned-vrgj4"
Sep 11 11:48:18.547710 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.546399    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7s55\" (UniqueName: \"kubernetes.io/projected/5f1597c3-cf23-4523-bb1b-9003cc6f4b2a-kube-api-access-j7s55\") pod \"iptables-alerter-pc2js\" (UID: \"5f1597c3-cf23-4523-bb1b-9003cc6f4b2a\") " pod="openshift-network-operator/iptables-alerter-pc2js"
Sep 11 11:48:18.547710 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.546425    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxl2q\" (UniqueName: \"kubernetes.io/projected/fa7be6d8-e93e-49af-8782-14c8080fef33-kube-api-access-sxl2q\") pod \"network-metrics-daemon-krxsv\" (UID: \"fa7be6d8-e93e-49af-8782-14c8080fef33\") " pod="openshift-multus/network-metrics-daemon-krxsv"
Sep 11 11:48:18.547710 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.546447    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2dd13752-864e-44e6-a1a5-986e623fbd7a-cni-binary-copy\") pod \"multus-lh5dd\" (UID: \"2dd13752-864e-44e6-a1a5-986e623fbd7a\") " pod="openshift-multus/multus-lh5dd"
Sep 11 11:48:18.547710 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.546496    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khcxv\" (UniqueName: \"kubernetes.io/projected/8fc69afd-c7cf-442c-8692-3ec71caffa8b-kube-api-access-khcxv\") pod \"ovnkube-node-2v285\" (UID: \"8fc69afd-c7cf-442c-8692-3ec71caffa8b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2v285"
Sep 11 11:48:18.547710 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.546522    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2dd13752-864e-44e6-a1a5-986e623fbd7a-multus-daemon-config\") pod \"multus-lh5dd\" (UID: \"2dd13752-864e-44e6-a1a5-986e623fbd7a\") " pod="openshift-multus/multus-lh5dd"
Sep 11 11:48:18.547710 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.546545    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9fbe0c28-1c37-4360-b183-dd216ce48906-etc-sysconfig\") pod \"tuned-vrgj4\" (UID: \"9fbe0c28-1c37-4360-b183-dd216ce48906\") " pod="openshift-cluster-node-tuning-operator/tuned-vrgj4"
Sep 11 11:48:18.547710 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.546569    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0f52f2ea-a695-422d-a16c-fa3db1f93041-host\") pod \"node-ca-t7cd7\" (UID: \"0f52f2ea-a695-422d-a16c-fa3db1f93041\") " pod="openshift-image-registry/node-ca-t7cd7"
Sep 11 11:48:18.547710 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.546816    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa7be6d8-e93e-49af-8782-14c8080fef33-metrics-certs\") pod \"network-metrics-daemon-krxsv\" (UID: \"fa7be6d8-e93e-49af-8782-14c8080fef33\") " pod="openshift-multus/network-metrics-daemon-krxsv"
Sep 11 11:48:18.547710 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.546844    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7872368e-69ce-4a5c-9773-e2d02b8f06ba-cni-binary-copy\") pod \"multus-additional-cni-plugins-4jthr\" (UID: \"7872368e-69ce-4a5c-9773-e2d02b8f06ba\") " pod="openshift-multus/multus-additional-cni-plugins-4jthr"
Sep 11 11:48:18.547710 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.546656    2579 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 11 11:48:18.547710 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.546977    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8fc69afd-c7cf-442c-8692-3ec71caffa8b-run-ovn\") pod \"ovnkube-node-2v285\" (UID: \"8fc69afd-c7cf-442c-8692-3ec71caffa8b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2v285"
Sep 11 11:48:18.547710 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.547010    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2dd13752-864e-44e6-a1a5-986e623fbd7a-multus-cni-dir\") pod \"multus-lh5dd\" (UID: \"2dd13752-864e-44e6-a1a5-986e623fbd7a\") " pod="openshift-multus/multus-lh5dd"
Sep 11 11:48:18.547710 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.547038    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2dd13752-864e-44e6-a1a5-986e623fbd7a-host-run-netns\") pod \"multus-lh5dd\" (UID: \"2dd13752-864e-44e6-a1a5-986e623fbd7a\") " pod="openshift-multus/multus-lh5dd"
Sep 11 11:48:18.547710 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.547068    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bcc619b6-bfb8-4cce-bf0c-b714377c7598-device-dir\") pod \"aws-ebs-csi-driver-node-kbzcr\" (UID: \"bcc619b6-bfb8-4cce-bf0c-b714377c7598\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kbzcr"
Sep 11 11:48:18.547710 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.547096    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7872368e-69ce-4a5c-9773-e2d02b8f06ba-cnibin\") pod \"multus-additional-cni-plugins-4jthr\" (UID: \"7872368e-69ce-4a5c-9773-e2d02b8f06ba\") " pod="openshift-multus/multus-additional-cni-plugins-4jthr"
Sep 11 11:48:18.548596 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.547124    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8fc69afd-c7cf-442c-8692-3ec71caffa8b-host-cni-netd\") pod \"ovnkube-node-2v285\" (UID: \"8fc69afd-c7cf-442c-8692-3ec71caffa8b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2v285"
Sep 11 11:48:18.548596 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.547152    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8fc69afd-c7cf-442c-8692-3ec71caffa8b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2v285\" (UID: \"8fc69afd-c7cf-442c-8692-3ec71caffa8b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2v285"
Sep 11 11:48:18.548596 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.547183    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2dd13752-864e-44e6-a1a5-986e623fbd7a-host-var-lib-kubelet\") pod \"multus-lh5dd\" (UID: \"2dd13752-864e-44e6-a1a5-986e623fbd7a\") " pod="openshift-multus/multus-lh5dd"
Sep 11 11:48:18.548596 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.547210    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8fc69afd-c7cf-442c-8692-3ec71caffa8b-var-lib-openvswitch\") pod \"ovnkube-node-2v285\" (UID: \"8fc69afd-c7cf-442c-8692-3ec71caffa8b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2v285"
Sep 11 11:48:18.548596 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.547243    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9fbe0c28-1c37-4360-b183-dd216ce48906-etc-systemd\") pod \"tuned-vrgj4\" (UID: \"9fbe0c28-1c37-4360-b183-dd216ce48906\") " pod="openshift-cluster-node-tuning-operator/tuned-vrgj4"
Sep 11 11:48:18.548596 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.547271    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5f1597c3-cf23-4523-bb1b-9003cc6f4b2a-host-slash\") pod \"iptables-alerter-pc2js\" (UID: \"5f1597c3-cf23-4523-bb1b-9003cc6f4b2a\") " pod="openshift-network-operator/iptables-alerter-pc2js"
Sep 11 11:48:18.548596 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.547300    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sllf7\" (UniqueName: \"kubernetes.io/projected/cdaded97-b0ac-4688-8721-813fc905c103-kube-api-access-sllf7\") pod \"node-resolver-2jqbl\" (UID: \"cdaded97-b0ac-4688-8721-813fc905c103\") " pod="openshift-dns/node-resolver-2jqbl"
Sep 11 11:48:18.548596 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.547327    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8fc69afd-c7cf-442c-8692-3ec71caffa8b-host-run-ovn-kubernetes\") pod \"ovnkube-node-2v285\" (UID: \"8fc69afd-c7cf-442c-8692-3ec71caffa8b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2v285"
Sep 11 11:48:18.548596 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.547360    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8fc69afd-c7cf-442c-8692-3ec71caffa8b-host-cni-bin\") pod \"ovnkube-node-2v285\" (UID: \"8fc69afd-c7cf-442c-8692-3ec71caffa8b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2v285"
Sep 11 11:48:18.548596 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.547387    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8fc69afd-c7cf-442c-8692-3ec71caffa8b-ovn-node-metrics-cert\") pod \"ovnkube-node-2v285\" (UID: \"8fc69afd-c7cf-442c-8692-3ec71caffa8b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2v285"
Sep 11 11:48:18.548596 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.547444    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7872368e-69ce-4a5c-9773-e2d02b8f06ba-cni-binary-copy\") pod \"multus-additional-cni-plugins-4jthr\" (UID: \"7872368e-69ce-4a5c-9773-e2d02b8f06ba\") " pod="openshift-multus/multus-additional-cni-plugins-4jthr"
Sep 11 11:48:18.548596 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.547535    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8fc69afd-c7cf-442c-8692-3ec71caffa8b-host-kubelet\") pod \"ovnkube-node-2v285\" (UID: \"8fc69afd-c7cf-442c-8692-3ec71caffa8b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2v285"
Sep 11 11:48:18.548596 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.547548    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bcc619b6-bfb8-4cce-bf0c-b714377c7598-device-dir\") pod \"aws-ebs-csi-driver-node-kbzcr\" (UID: \"bcc619b6-bfb8-4cce-bf0c-b714377c7598\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kbzcr"
Sep 11 11:48:18.548596 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.547571    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9fbe0c28-1c37-4360-b183-dd216ce48906-etc-modprobe-d\") pod \"tuned-vrgj4\" (UID: \"9fbe0c28-1c37-4360-b183-dd216ce48906\") " pod="openshift-cluster-node-tuning-operator/tuned-vrgj4"
Sep 11 11:48:18.548596 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.547588    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7872368e-69ce-4a5c-9773-e2d02b8f06ba-cnibin\") pod \"multus-additional-cni-plugins-4jthr\" (UID: \"7872368e-69ce-4a5c-9773-e2d02b8f06ba\") " pod="openshift-multus/multus-additional-cni-plugins-4jthr"
Sep 11 11:48:18.548596 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.547600    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9fbe0c28-1c37-4360-b183-dd216ce48906-lib-modules\") pod \"tuned-vrgj4\" (UID: \"9fbe0c28-1c37-4360-b183-dd216ce48906\") " pod="openshift-cluster-node-tuning-operator/tuned-vrgj4"
Sep 11 11:48:18.548596 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.547629    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9fbe0c28-1c37-4360-b183-dd216ce48906-var-lib-kubelet\") pod \"tuned-vrgj4\" (UID: \"9fbe0c28-1c37-4360-b183-dd216ce48906\") " pod="openshift-cluster-node-tuning-operator/tuned-vrgj4"
Sep 11 11:48:18.549494 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.547655    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvf85\" (UniqueName: \"kubernetes.io/projected/9fbe0c28-1c37-4360-b183-dd216ce48906-kube-api-access-xvf85\") pod \"tuned-vrgj4\" (UID: \"9fbe0c28-1c37-4360-b183-dd216ce48906\") " pod="openshift-cluster-node-tuning-operator/tuned-vrgj4"
Sep 11 11:48:18.549494 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.547642    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9fbe0c28-1c37-4360-b183-dd216ce48906-etc-systemd\") pod \"tuned-vrgj4\" (UID: \"9fbe0c28-1c37-4360-b183-dd216ce48906\") " pod="openshift-cluster-node-tuning-operator/tuned-vrgj4"
Sep 11 11:48:18.549494 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.547683    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5f1597c3-cf23-4523-bb1b-9003cc6f4b2a-iptables-alerter-script\") pod \"iptables-alerter-pc2js\" (UID: \"5f1597c3-cf23-4523-bb1b-9003cc6f4b2a\") " pod="openshift-network-operator/iptables-alerter-pc2js"
Sep 11 11:48:18.549494 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.547710    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8fc69afd-c7cf-442c-8692-3ec71caffa8b-node-log\") pod \"ovnkube-node-2v285\" (UID: \"8fc69afd-c7cf-442c-8692-3ec71caffa8b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2v285"
Sep 11 11:48:18.549494 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.547722    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5f1597c3-cf23-4523-bb1b-9003cc6f4b2a-host-slash\") pod \"iptables-alerter-pc2js\" (UID: \"5f1597c3-cf23-4523-bb1b-9003cc6f4b2a\") " pod="openshift-network-operator/iptables-alerter-pc2js"
Sep 11 11:48:18.549494 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.547737    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2dd13752-864e-44e6-a1a5-986e623fbd7a-cnibin\") pod \"multus-lh5dd\" (UID: \"2dd13752-864e-44e6-a1a5-986e623fbd7a\") " pod="openshift-multus/multus-lh5dd"
Sep 11 11:48:18.549494 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.547772    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8fc69afd-c7cf-442c-8692-3ec71caffa8b-host-run-netns\") pod \"ovnkube-node-2v285\" (UID: \"8fc69afd-c7cf-442c-8692-3ec71caffa8b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2v285"
Sep 11 11:48:18.549494 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.547800    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bcc619b6-bfb8-4cce-bf0c-b714377c7598-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kbzcr\" (UID: \"bcc619b6-bfb8-4cce-bf0c-b714377c7598\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kbzcr"
Sep 11 11:48:18.549494 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.547804    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9fbe0c28-1c37-4360-b183-dd216ce48906-etc-sysctl-d\") pod \"tuned-vrgj4\" (UID: \"9fbe0c28-1c37-4360-b183-dd216ce48906\") " pod="openshift-cluster-node-tuning-operator/tuned-vrgj4"
Sep 11 11:48:18.549494 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:18.547830    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 11:48:18.549494 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.547903    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9fbe0c28-1c37-4360-b183-dd216ce48906-etc-sysconfig\") pod \"tuned-vrgj4\" (UID: \"9fbe0c28-1c37-4360-b183-dd216ce48906\") " pod="openshift-cluster-node-tuning-operator/tuned-vrgj4"
Sep 11 11:48:18.549494 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.547971    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0f52f2ea-a695-422d-a16c-fa3db1f93041-host\") pod \"node-ca-t7cd7\" (UID: \"0f52f2ea-a695-422d-a16c-fa3db1f93041\") " pod="openshift-image-registry/node-ca-t7cd7"
Sep 11 11:48:18.549494 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.547972    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9fbe0c28-1c37-4360-b183-dd216ce48906-etc-sysctl-d\") pod \"tuned-vrgj4\" (UID: \"9fbe0c28-1c37-4360-b183-dd216ce48906\") " pod="openshift-cluster-node-tuning-operator/tuned-vrgj4"
Sep 11 11:48:18.549494 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.548009    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9fbe0c28-1c37-4360-b183-dd216ce48906-var-lib-kubelet\") pod \"tuned-vrgj4\" (UID: \"9fbe0c28-1c37-4360-b183-dd216ce48906\") " pod="openshift-cluster-node-tuning-operator/tuned-vrgj4"
Sep 11 11:48:18.549494 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.548009    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0f52f2ea-a695-422d-a16c-fa3db1f93041-serviceca\") pod \"node-ca-t7cd7\" (UID: \"0f52f2ea-a695-422d-a16c-fa3db1f93041\") " pod="openshift-image-registry/node-ca-t7cd7"
Sep 11 11:48:18.549494 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:18.548060    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa7be6d8-e93e-49af-8782-14c8080fef33-metrics-certs podName:fa7be6d8-e93e-49af-8782-14c8080fef33 nodeName:}" failed. No retries permitted until 2026-09-11 11:48:19.048033009 +0000 UTC m=+2.945474757 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fa7be6d8-e93e-49af-8782-14c8080fef33-metrics-certs") pod "network-metrics-daemon-krxsv" (UID: "fa7be6d8-e93e-49af-8782-14c8080fef33") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 11:48:18.549494 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.548075    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9fbe0c28-1c37-4360-b183-dd216ce48906-host\") pod \"tuned-vrgj4\" (UID: \"9fbe0c28-1c37-4360-b183-dd216ce48906\") " pod="openshift-cluster-node-tuning-operator/tuned-vrgj4"
Sep 11 11:48:18.550359 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.548080    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bcc619b6-bfb8-4cce-bf0c-b714377c7598-sys-fs\") pod \"aws-ebs-csi-driver-node-kbzcr\" (UID: \"bcc619b6-bfb8-4cce-bf0c-b714377c7598\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kbzcr"
Sep 11 11:48:18.550359 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.548113    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7872368e-69ce-4a5c-9773-e2d02b8f06ba-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4jthr\" (UID: \"7872368e-69ce-4a5c-9773-e2d02b8f06ba\") " pod="openshift-multus/multus-additional-cni-plugins-4jthr"
Sep 11 11:48:18.550359 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.548198    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9fbe0c28-1c37-4360-b183-dd216ce48906-etc-sysctl-conf\") pod \"tuned-vrgj4\" (UID: \"9fbe0c28-1c37-4360-b183-dd216ce48906\") " pod="openshift-cluster-node-tuning-operator/tuned-vrgj4"
Sep 11 11:48:18.550359 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.548284    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7872368e-69ce-4a5c-9773-e2d02b8f06ba-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4jthr\" (UID: \"7872368e-69ce-4a5c-9773-e2d02b8f06ba\") " pod="openshift-multus/multus-additional-cni-plugins-4jthr"
Sep 11 11:48:18.550359 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.548358    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9fbe0c28-1c37-4360-b183-dd216ce48906-etc-modprobe-d\") pod \"tuned-vrgj4\" (UID: \"9fbe0c28-1c37-4360-b183-dd216ce48906\") " pod="openshift-cluster-node-tuning-operator/tuned-vrgj4"
Sep 11 11:48:18.550359 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.548082    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9fbe0c28-1c37-4360-b183-dd216ce48906-lib-modules\") pod \"tuned-vrgj4\" (UID: \"9fbe0c28-1c37-4360-b183-dd216ce48906\") " pod="openshift-cluster-node-tuning-operator/tuned-vrgj4"
Sep 11 11:48:18.550359 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.548490    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7872368e-69ce-4a5c-9773-e2d02b8f06ba-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4jthr\" (UID: \"7872368e-69ce-4a5c-9773-e2d02b8f06ba\") " pod="openshift-multus/multus-additional-cni-plugins-4jthr"
Sep 11 11:48:18.550359 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.548507    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bcc619b6-bfb8-4cce-bf0c-b714377c7598-sys-fs\") pod \"aws-ebs-csi-driver-node-kbzcr\" (UID: \"bcc619b6-bfb8-4cce-bf0c-b714377c7598\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kbzcr"
Sep 11 11:48:18.550359 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.548835    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5f1597c3-cf23-4523-bb1b-9003cc6f4b2a-iptables-alerter-script\") pod \"iptables-alerter-pc2js\" (UID: \"5f1597c3-cf23-4523-bb1b-9003cc6f4b2a\") " pod="openshift-network-operator/iptables-alerter-pc2js"
Sep 11 11:48:18.553185 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.553125    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9fbe0c28-1c37-4360-b183-dd216ce48906-tmp\") pod \"tuned-vrgj4\" (UID: \"9fbe0c28-1c37-4360-b183-dd216ce48906\") " pod="openshift-cluster-node-tuning-operator/tuned-vrgj4"
Sep 11 11:48:18.554013 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.553993    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9fbe0c28-1c37-4360-b183-dd216ce48906-etc-tuned\") pod \"tuned-vrgj4\" (UID: \"9fbe0c28-1c37-4360-b183-dd216ce48906\") " pod="openshift-cluster-node-tuning-operator/tuned-vrgj4"
Sep 11 11:48:18.555700 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:18.555660    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 11:48:18.555700 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:18.555689    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 11:48:18.555700 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:18.555702    2579 projected.go:196] Error preparing data for projected volume kube-api-access-5qbg9 for pod openshift-network-diagnostics/network-check-target-dqc42: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 11:48:18.555999 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:18.555773    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/711f1752-03c4-4f9a-bb1b-cb7bf8cac9cd-kube-api-access-5qbg9 podName:711f1752-03c4-4f9a-bb1b-cb7bf8cac9cd nodeName:}" failed. No retries permitted until 2026-09-11 11:48:19.05575194 +0000 UTC m=+2.953193691 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-5qbg9" (UniqueName: "kubernetes.io/projected/711f1752-03c4-4f9a-bb1b-cb7bf8cac9cd-kube-api-access-5qbg9") pod "network-check-target-dqc42" (UID: "711f1752-03c4-4f9a-bb1b-cb7bf8cac9cd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 11:48:18.559948 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.559923    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zp5xs\" (UniqueName: \"kubernetes.io/projected/7872368e-69ce-4a5c-9773-e2d02b8f06ba-kube-api-access-zp5xs\") pod \"multus-additional-cni-plugins-4jthr\" (UID: \"7872368e-69ce-4a5c-9773-e2d02b8f06ba\") " pod="openshift-multus/multus-additional-cni-plugins-4jthr"
Sep 11 11:48:18.562683 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.562658    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-75hnp\" (UniqueName: \"kubernetes.io/projected/bcc619b6-bfb8-4cce-bf0c-b714377c7598-kube-api-access-75hnp\") pod \"aws-ebs-csi-driver-node-kbzcr\" (UID: \"bcc619b6-bfb8-4cce-bf0c-b714377c7598\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kbzcr"
Sep 11 11:48:18.564896 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.564868    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7s55\" (UniqueName: \"kubernetes.io/projected/5f1597c3-cf23-4523-bb1b-9003cc6f4b2a-kube-api-access-j7s55\") pod \"iptables-alerter-pc2js\" (UID: \"5f1597c3-cf23-4523-bb1b-9003cc6f4b2a\") " pod="openshift-network-operator/iptables-alerter-pc2js"
Sep 11 11:48:18.564997 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.564876    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-td5q7\" (UniqueName: \"kubernetes.io/projected/0f52f2ea-a695-422d-a16c-fa3db1f93041-kube-api-access-td5q7\") pod \"node-ca-t7cd7\" (UID: \"0f52f2ea-a695-422d-a16c-fa3db1f93041\") " pod="openshift-image-registry/node-ca-t7cd7"
Sep 11 11:48:18.566218 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.566194    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxl2q\" (UniqueName: \"kubernetes.io/projected/fa7be6d8-e93e-49af-8782-14c8080fef33-kube-api-access-sxl2q\") pod \"network-metrics-daemon-krxsv\" (UID: \"fa7be6d8-e93e-49af-8782-14c8080fef33\") " pod="openshift-multus/network-metrics-daemon-krxsv"
Sep 11 11:48:18.566618 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.566563    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvf85\" (UniqueName: \"kubernetes.io/projected/9fbe0c28-1c37-4360-b183-dd216ce48906-kube-api-access-xvf85\") pod \"tuned-vrgj4\" (UID: \"9fbe0c28-1c37-4360-b183-dd216ce48906\") " pod="openshift-cluster-node-tuning-operator/tuned-vrgj4"
Sep 11 11:48:18.576778 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.576712    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-145.ec2.internal" event={"ID":"0a88adf83eef5bb51fa329190cac6245","Type":"ContainerStarted","Data":"e4cd1382466494bb5697c0281827d321bc8698c471c17fb0c719696feaa38689"}
Sep 11 11:48:18.578736 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.578693    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-145.ec2.internal" event={"ID":"3d9525f1a3218f8a340d89c0b17b6cdb","Type":"ContainerStarted","Data":"e8a647904c1c642553160406855f287d192e8e28a327ce27be029ae8166067be"}
Sep 11 11:48:18.649431 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.649069    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2dd13752-864e-44e6-a1a5-986e623fbd7a-cni-binary-copy\") pod \"multus-lh5dd\" (UID: \"2dd13752-864e-44e6-a1a5-986e623fbd7a\") " pod="openshift-multus/multus-lh5dd"
Sep 11 11:48:18.649431 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.649118    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khcxv\" (UniqueName: \"kubernetes.io/projected/8fc69afd-c7cf-442c-8692-3ec71caffa8b-kube-api-access-khcxv\") pod \"ovnkube-node-2v285\" (UID: \"8fc69afd-c7cf-442c-8692-3ec71caffa8b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2v285"
Sep 11 11:48:18.649431 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.649145    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2dd13752-864e-44e6-a1a5-986e623fbd7a-multus-daemon-config\") pod \"multus-lh5dd\" (UID: \"2dd13752-864e-44e6-a1a5-986e623fbd7a\") " pod="openshift-multus/multus-lh5dd"
Sep 11 11:48:18.649431 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.649190    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8fc69afd-c7cf-442c-8692-3ec71caffa8b-run-ovn\") pod \"ovnkube-node-2v285\" (UID: \"8fc69afd-c7cf-442c-8692-3ec71caffa8b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2v285"
Sep 11 11:48:18.649431 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.649212    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2dd13752-864e-44e6-a1a5-986e623fbd7a-multus-cni-dir\") pod \"multus-lh5dd\" (UID: \"2dd13752-864e-44e6-a1a5-986e623fbd7a\") " pod="openshift-multus/multus-lh5dd"
Sep 11 11:48:18.649431 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.649235    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2dd13752-864e-44e6-a1a5-986e623fbd7a-host-run-netns\") pod \"multus-lh5dd\" (UID: \"2dd13752-864e-44e6-a1a5-986e623fbd7a\") " pod="openshift-multus/multus-lh5dd"
Sep 11 11:48:18.649431 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.649259    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8fc69afd-c7cf-442c-8692-3ec71caffa8b-host-cni-netd\") pod \"ovnkube-node-2v285\" (UID: \"8fc69afd-c7cf-442c-8692-3ec71caffa8b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2v285"
Sep 11 11:48:18.649431 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.649283    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8fc69afd-c7cf-442c-8692-3ec71caffa8b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2v285\" (UID: \"8fc69afd-c7cf-442c-8692-3ec71caffa8b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2v285"
Sep 11 11:48:18.649431 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.649319    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2dd13752-864e-44e6-a1a5-986e623fbd7a-host-var-lib-kubelet\") pod \"multus-lh5dd\" (UID: \"2dd13752-864e-44e6-a1a5-986e623fbd7a\") " pod="openshift-multus/multus-lh5dd"
Sep 11 11:48:18.649431 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.649344    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8fc69afd-c7cf-442c-8692-3ec71caffa8b-var-lib-openvswitch\") pod \"ovnkube-node-2v285\" (UID: \"8fc69afd-c7cf-442c-8692-3ec71caffa8b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2v285"
Sep 11 11:48:18.649431 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.649373    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sllf7\" (UniqueName: \"kubernetes.io/projected/cdaded97-b0ac-4688-8721-813fc905c103-kube-api-access-sllf7\") pod \"node-resolver-2jqbl\" (UID: \"cdaded97-b0ac-4688-8721-813fc905c103\") " pod="openshift-dns/node-resolver-2jqbl"
Sep 11 11:48:18.649431 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.649397    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8fc69afd-c7cf-442c-8692-3ec71caffa8b-host-run-ovn-kubernetes\") pod \"ovnkube-node-2v285\" (UID: \"8fc69afd-c7cf-442c-8692-3ec71caffa8b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2v285"
Sep 11 11:48:18.649431 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.649426    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8fc69afd-c7cf-442c-8692-3ec71caffa8b-host-cni-bin\") pod \"ovnkube-node-2v285\" (UID: \"8fc69afd-c7cf-442c-8692-3ec71caffa8b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2v285"
Sep 11 11:48:18.650101 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.649451    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8fc69afd-c7cf-442c-8692-3ec71caffa8b-ovn-node-metrics-cert\") pod \"ovnkube-node-2v285\" (UID: \"8fc69afd-c7cf-442c-8692-3ec71caffa8b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2v285"
Sep 11 11:48:18.650101 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.649511    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8fc69afd-c7cf-442c-8692-3ec71caffa8b-host-kubelet\") pod \"ovnkube-node-2v285\" (UID: \"8fc69afd-c7cf-442c-8692-3ec71caffa8b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2v285"
Sep 11 11:48:18.650101 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.649544    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8fc69afd-c7cf-442c-8692-3ec71caffa8b-node-log\") pod \"ovnkube-node-2v285\" (UID: \"8fc69afd-c7cf-442c-8692-3ec71caffa8b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2v285"
Sep 11 11:48:18.650101 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.649571    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2dd13752-864e-44e6-a1a5-986e623fbd7a-cnibin\") pod \"multus-lh5dd\" (UID: \"2dd13752-864e-44e6-a1a5-986e623fbd7a\") " pod="openshift-multus/multus-lh5dd"
Sep 11 11:48:18.650101 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.649596    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8fc69afd-c7cf-442c-8692-3ec71caffa8b-host-run-netns\") pod \"ovnkube-node-2v285\" (UID: \"8fc69afd-c7cf-442c-8692-3ec71caffa8b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2v285"
Sep 11 11:48:18.650101 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.649627    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2dd13752-864e-44e6-a1a5-986e623fbd7a-host-run-k8s-cni-cncf-io\") pod \"multus-lh5dd\" (UID: \"2dd13752-864e-44e6-a1a5-986e623fbd7a\") " pod="openshift-multus/multus-lh5dd"
Sep 11 11:48:18.650101 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.649655    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cdaded97-b0ac-4688-8721-813fc905c103-tmp-dir\") pod \"node-resolver-2jqbl\" (UID: \"cdaded97-b0ac-4688-8721-813fc905c103\") " pod="openshift-dns/node-resolver-2jqbl"
Sep 11 11:48:18.650101 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.649680    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8fc69afd-c7cf-442c-8692-3ec71caffa8b-host-slash\") pod \"ovnkube-node-2v285\" (UID: \"8fc69afd-c7cf-442c-8692-3ec71caffa8b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2v285"
Sep 11 11:48:18.650101 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.649707    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2dd13752-864e-44e6-a1a5-986e623fbd7a-multus-socket-dir-parent\") pod \"multus-lh5dd\" (UID: \"2dd13752-864e-44e6-a1a5-986e623fbd7a\") " pod="openshift-multus/multus-lh5dd"
Sep 11 11:48:18.650101 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.649732    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2dd13752-864e-44e6-a1a5-986e623fbd7a-host-var-lib-cni-bin\") pod \"multus-lh5dd\" (UID: \"2dd13752-864e-44e6-a1a5-986e623fbd7a\") " pod="openshift-multus/multus-lh5dd"
Sep 11 11:48:18.650101 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.649757    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2dd13752-864e-44e6-a1a5-986e623fbd7a-hostroot\") pod \"multus-lh5dd\" (UID: \"2dd13752-864e-44e6-a1a5-986e623fbd7a\") " pod="openshift-multus/multus-lh5dd"
Sep 11 11:48:18.650101 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.649773    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2dd13752-864e-44e6-a1a5-986e623fbd7a-cni-binary-copy\") pod \"multus-lh5dd\" (UID: \"2dd13752-864e-44e6-a1a5-986e623fbd7a\") " pod="openshift-multus/multus-lh5dd"
Sep 11 11:48:18.650101 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.649780    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2dd13752-864e-44e6-a1a5-986e623fbd7a-host-run-multus-certs\") pod \"multus-lh5dd\" (UID: \"2dd13752-864e-44e6-a1a5-986e623fbd7a\") " pod="openshift-multus/multus-lh5dd"
Sep 11 11:48:18.650101 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.649806    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2dd13752-864e-44e6-a1a5-986e623fbd7a-host-var-lib-cni-multus\") pod \"multus-lh5dd\" (UID: \"2dd13752-864e-44e6-a1a5-986e623fbd7a\") " pod="openshift-multus/multus-lh5dd"
Sep 11 11:48:18.650101 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.649831    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2dd13752-864e-44e6-a1a5-986e623fbd7a-etc-kubernetes\") pod \"multus-lh5dd\" (UID: \"2dd13752-864e-44e6-a1a5-986e623fbd7a\") " pod="openshift-multus/multus-lh5dd"
Sep 11 11:48:18.650101 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.649860    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/70994cc4-c221-4ab7-99ec-aed7c86955de-dbus\") pod \"global-pull-secret-syncer-79ltx\" (UID: \"70994cc4-c221-4ab7-99ec-aed7c86955de\") " pod="kube-system/global-pull-secret-syncer-79ltx"
Sep 11 11:48:18.650101 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.649883    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8fc69afd-c7cf-442c-8692-3ec71caffa8b-ovnkube-config\") pod \"ovnkube-node-2v285\" (UID: \"8fc69afd-c7cf-442c-8692-3ec71caffa8b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2v285"
Sep 11 11:48:18.650101 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.649908    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zp4rk\" (UniqueName: \"kubernetes.io/projected/2dd13752-864e-44e6-a1a5-986e623fbd7a-kube-api-access-zp4rk\") pod \"multus-lh5dd\" (UID: \"2dd13752-864e-44e6-a1a5-986e623fbd7a\") " pod="openshift-multus/multus-lh5dd"
Sep 11 11:48:18.651014 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.649933    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/70994cc4-c221-4ab7-99ec-aed7c86955de-kubelet-config\") pod \"global-pull-secret-syncer-79ltx\" (UID: \"70994cc4-c221-4ab7-99ec-aed7c86955de\") " pod="kube-system/global-pull-secret-syncer-79ltx"
Sep 11 11:48:18.651014 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.649956    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/70994cc4-c221-4ab7-99ec-aed7c86955de-original-pull-secret\") pod \"global-pull-secret-syncer-79ltx\" (UID: \"70994cc4-c221-4ab7-99ec-aed7c86955de\") " pod="kube-system/global-pull-secret-syncer-79ltx"
Sep 11 11:48:18.651014 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.649984    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8fc69afd-c7cf-442c-8692-3ec71caffa8b-run-systemd\") pod \"ovnkube-node-2v285\" (UID: \"8fc69afd-c7cf-442c-8692-3ec71caffa8b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2v285"
Sep 11 11:48:18.651014 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.650023    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8fc69afd-c7cf-442c-8692-3ec71caffa8b-systemd-units\") pod \"ovnkube-node-2v285\" (UID: \"8fc69afd-c7cf-442c-8692-3ec71caffa8b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2v285"
Sep 11 11:48:18.651014 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.650046    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8fc69afd-c7cf-442c-8692-3ec71caffa8b-ovnkube-script-lib\") pod \"ovnkube-node-2v285\" (UID: \"8fc69afd-c7cf-442c-8692-3ec71caffa8b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2v285"
Sep 11 11:48:18.651014 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.650073    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8fc69afd-c7cf-442c-8692-3ec71caffa8b-run-openvswitch\") pod \"ovnkube-node-2v285\" (UID: \"8fc69afd-c7cf-442c-8692-3ec71caffa8b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2v285"
Sep 11 11:48:18.651014 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.650105    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8fc69afd-c7cf-442c-8692-3ec71caffa8b-env-overrides\") pod \"ovnkube-node-2v285\" (UID: \"8fc69afd-c7cf-442c-8692-3ec71caffa8b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2v285"
Sep 11 11:48:18.651014 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.650130    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cdaded97-b0ac-4688-8721-813fc905c103-hosts-file\") pod \"node-resolver-2jqbl\" (UID: \"cdaded97-b0ac-4688-8721-813fc905c103\") " pod="openshift-dns/node-resolver-2jqbl"
Sep 11 11:48:18.651014 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.650155    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8fc69afd-c7cf-442c-8692-3ec71caffa8b-etc-openvswitch\") pod \"ovnkube-node-2v285\" (UID: \"8fc69afd-c7cf-442c-8692-3ec71caffa8b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2v285"
Sep 11 11:48:18.651014 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.650187    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/43cde4a1-c7f9-443d-9f2f-0ab809b5d25a-agent-certs\") pod \"konnectivity-agent-dxf9l\" (UID: \"43cde4a1-c7f9-443d-9f2f-0ab809b5d25a\") " pod="kube-system/konnectivity-agent-dxf9l"
Sep 11 11:48:18.651014 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.650212    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8fc69afd-c7cf-442c-8692-3ec71caffa8b-log-socket\") pod \"ovnkube-node-2v285\" (UID: \"8fc69afd-c7cf-442c-8692-3ec71caffa8b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2v285"
Sep 11 11:48:18.651014 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.650238    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/43cde4a1-c7f9-443d-9f2f-0ab809b5d25a-konnectivity-ca\") pod \"konnectivity-agent-dxf9l\" (UID: \"43cde4a1-c7f9-443d-9f2f-0ab809b5d25a\") " pod="kube-system/konnectivity-agent-dxf9l"
Sep 11 11:48:18.651014 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.650282    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2dd13752-864e-44e6-a1a5-986e623fbd7a-system-cni-dir\") pod \"multus-lh5dd\" (UID: \"2dd13752-864e-44e6-a1a5-986e623fbd7a\") " pod="openshift-multus/multus-lh5dd"
Sep 11 11:48:18.651014 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.650309    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2dd13752-864e-44e6-a1a5-986e623fbd7a-multus-conf-dir\") pod \"multus-lh5dd\" (UID: \"2dd13752-864e-44e6-a1a5-986e623fbd7a\") " pod="openshift-multus/multus-lh5dd"
Sep 11 11:48:18.651014 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.650402    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2dd13752-864e-44e6-a1a5-986e623fbd7a-multus-conf-dir\") pod \"multus-lh5dd\" (UID: \"2dd13752-864e-44e6-a1a5-986e623fbd7a\") " pod="openshift-multus/multus-lh5dd"
Sep 11 11:48:18.651014 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.650559    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2dd13752-864e-44e6-a1a5-986e623fbd7a-hostroot\") pod \"multus-lh5dd\" (UID: \"2dd13752-864e-44e6-a1a5-986e623fbd7a\") " pod="openshift-multus/multus-lh5dd"
Sep 11 11:48:18.651014 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.650580    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2dd13752-864e-44e6-a1a5-986e623fbd7a-host-var-lib-cni-bin\") pod \"multus-lh5dd\" (UID: \"2dd13752-864e-44e6-a1a5-986e623fbd7a\") " pod="openshift-multus/multus-lh5dd"
Sep 11 11:48:18.651014 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.650608    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8fc69afd-c7cf-442c-8692-3ec71caffa8b-run-systemd\") pod \"ovnkube-node-2v285\" (UID: \"8fc69afd-c7cf-442c-8692-3ec71caffa8b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2v285"
Sep 11 11:48:18.651551 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.650654    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2dd13752-864e-44e6-a1a5-986e623fbd7a-host-run-multus-certs\") pod \"multus-lh5dd\" (UID: \"2dd13752-864e-44e6-a1a5-986e623fbd7a\") " pod="openshift-multus/multus-lh5dd"
Sep 11 11:48:18.651551 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.650684    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8fc69afd-c7cf-442c-8692-3ec71caffa8b-systemd-units\") pod \"ovnkube-node-2v285\" (UID: \"8fc69afd-c7cf-442c-8692-3ec71caffa8b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2v285"
Sep 11 11:48:18.651551 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.650699    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2dd13752-864e-44e6-a1a5-986e623fbd7a-host-var-lib-cni-multus\") pod \"multus-lh5dd\" (UID: \"2dd13752-864e-44e6-a1a5-986e623fbd7a\") " pod="openshift-multus/multus-lh5dd"
Sep 11 11:48:18.651551 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.650739    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/70994cc4-c221-4ab7-99ec-aed7c86955de-dbus\") pod \"global-pull-secret-syncer-79ltx\" (UID: \"70994cc4-c221-4ab7-99ec-aed7c86955de\") " pod="kube-system/global-pull-secret-syncer-79ltx"
Sep 11 11:48:18.651551 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.650745    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2dd13752-864e-44e6-a1a5-986e623fbd7a-etc-kubernetes\") pod \"multus-lh5dd\" (UID: \"2dd13752-864e-44e6-a1a5-986e623fbd7a\") " pod="openshift-multus/multus-lh5dd"
Sep 11 11:48:18.651551 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.650791    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/70994cc4-c221-4ab7-99ec-aed7c86955de-kubelet-config\") pod \"global-pull-secret-syncer-79ltx\" (UID: \"70994cc4-c221-4ab7-99ec-aed7c86955de\") " pod="kube-system/global-pull-secret-syncer-79ltx"
Sep 11 11:48:18.651551 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:18.651009    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 11:48:18.651551 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.651028    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8fc69afd-c7cf-442c-8692-3ec71caffa8b-host-cni-bin\") pod \"ovnkube-node-2v285\" (UID: \"8fc69afd-c7cf-442c-8692-3ec71caffa8b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2v285"
Sep 11 11:48:18.651551 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.651075    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8fc69afd-c7cf-442c-8692-3ec71caffa8b-run-ovn\") pod \"ovnkube-node-2v285\" (UID: \"8fc69afd-c7cf-442c-8692-3ec71caffa8b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2v285"
Sep 11 11:48:18.651551 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:18.651088    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70994cc4-c221-4ab7-99ec-aed7c86955de-original-pull-secret podName:70994cc4-c221-4ab7-99ec-aed7c86955de nodeName:}" failed. No retries permitted until 2026-09-11 11:48:19.151069218 +0000 UTC m=+3.048510956 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/70994cc4-c221-4ab7-99ec-aed7c86955de-original-pull-secret") pod "global-pull-secret-syncer-79ltx" (UID: "70994cc4-c221-4ab7-99ec-aed7c86955de") : object "kube-system"/"original-pull-secret" not registered
Sep 11 11:48:18.651551 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.651111    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8fc69afd-c7cf-442c-8692-3ec71caffa8b-ovnkube-config\") pod \"ovnkube-node-2v285\" (UID: \"8fc69afd-c7cf-442c-8692-3ec71caffa8b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2v285"
Sep 11 11:48:18.651551 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.651136    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2dd13752-864e-44e6-a1a5-986e623fbd7a-multus-cni-dir\") pod \"multus-lh5dd\" (UID: \"2dd13752-864e-44e6-a1a5-986e623fbd7a\") " pod="openshift-multus/multus-lh5dd"
Sep 11 11:48:18.651551 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.651163    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2dd13752-864e-44e6-a1a5-986e623fbd7a-multus-daemon-config\") pod \"multus-lh5dd\" (UID: \"2dd13752-864e-44e6-a1a5-986e623fbd7a\") " pod="openshift-multus/multus-lh5dd"
Sep 11 11:48:18.651551 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.651175    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2dd13752-864e-44e6-a1a5-986e623fbd7a-host-var-lib-kubelet\") pod \"multus-lh5dd\" (UID: \"2dd13752-864e-44e6-a1a5-986e623fbd7a\") " pod="openshift-multus/multus-lh5dd"
Sep 11 11:48:18.651551 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.651180    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8fc69afd-c7cf-442c-8692-3ec71caffa8b-var-lib-openvswitch\") pod \"ovnkube-node-2v285\" (UID: \"8fc69afd-c7cf-442c-8692-3ec71caffa8b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2v285"
Sep 11 11:48:18.651551 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.651221    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2dd13752-864e-44e6-a1a5-986e623fbd7a-host-run-netns\") pod \"multus-lh5dd\" (UID: \"2dd13752-864e-44e6-a1a5-986e623fbd7a\") " pod="openshift-multus/multus-lh5dd"
Sep 11 11:48:18.651551 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.651232    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2dd13752-864e-44e6-a1a5-986e623fbd7a-cnibin\") pod \"multus-lh5dd\" (UID: \"2dd13752-864e-44e6-a1a5-986e623fbd7a\") " pod="openshift-multus/multus-lh5dd"
Sep 11 11:48:18.651551 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.651258    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8fc69afd-c7cf-442c-8692-3ec71caffa8b-ovnkube-script-lib\") pod \"ovnkube-node-2v285\" (UID: \"8fc69afd-c7cf-442c-8692-3ec71caffa8b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2v285"
Sep 11 11:48:18.652418 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.651275    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8fc69afd-c7cf-442c-8692-3ec71caffa8b-host-run-ovn-kubernetes\") pod \"ovnkube-node-2v285\" (UID: \"8fc69afd-c7cf-442c-8692-3ec71caffa8b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2v285"
Sep 11 11:48:18.652418 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.651277    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8fc69afd-c7cf-442c-8692-3ec71caffa8b-host-slash\") pod \"ovnkube-node-2v285\" (UID: \"8fc69afd-c7cf-442c-8692-3ec71caffa8b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2v285"
Sep 11 11:48:18.652418 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.651292    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8fc69afd-c7cf-442c-8692-3ec71caffa8b-host-cni-netd\") pod \"ovnkube-node-2v285\" (UID: \"8fc69afd-c7cf-442c-8692-3ec71caffa8b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2v285"
Sep 11 11:48:18.652418 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.651312    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8fc69afd-c7cf-442c-8692-3ec71caffa8b-run-openvswitch\") pod \"ovnkube-node-2v285\" (UID: \"8fc69afd-c7cf-442c-8692-3ec71caffa8b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2v285"
Sep 11 11:48:18.652418 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.651329    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2dd13752-864e-44e6-a1a5-986e623fbd7a-multus-socket-dir-parent\") pod \"multus-lh5dd\" (UID: \"2dd13752-864e-44e6-a1a5-986e623fbd7a\") " pod="openshift-multus/multus-lh5dd"
Sep 11 11:48:18.652418 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.651334    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8fc69afd-c7cf-442c-8692-3ec71caffa8b-node-log\") pod \"ovnkube-node-2v285\" (UID: \"8fc69afd-c7cf-442c-8692-3ec71caffa8b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2v285"
Sep 11 11:48:18.652418 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.651341    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8fc69afd-c7cf-442c-8692-3ec71caffa8b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2v285\" (UID: \"8fc69afd-c7cf-442c-8692-3ec71caffa8b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2v285"
Sep 11 11:48:18.652418 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.651374    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8fc69afd-c7cf-442c-8692-3ec71caffa8b-host-kubelet\") pod \"ovnkube-node-2v285\" (UID: \"8fc69afd-c7cf-442c-8692-3ec71caffa8b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2v285"
Sep 11 11:48:18.652418 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.651382    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8fc69afd-c7cf-442c-8692-3ec71caffa8b-log-socket\") pod \"ovnkube-node-2v285\" (UID: \"8fc69afd-c7cf-442c-8692-3ec71caffa8b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2v285"
Sep 11 11:48:18.652418 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.651398    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cdaded97-b0ac-4688-8721-813fc905c103-hosts-file\") pod \"node-resolver-2jqbl\" (UID: \"cdaded97-b0ac-4688-8721-813fc905c103\") " pod="openshift-dns/node-resolver-2jqbl"
Sep 11 11:48:18.652418 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.651420    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8fc69afd-c7cf-442c-8692-3ec71caffa8b-etc-openvswitch\") pod \"ovnkube-node-2v285\" (UID: \"8fc69afd-c7cf-442c-8692-3ec71caffa8b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2v285"
Sep 11 11:48:18.652418 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.651461    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8fc69afd-c7cf-442c-8692-3ec71caffa8b-host-run-netns\") pod \"ovnkube-node-2v285\" (UID: \"8fc69afd-c7cf-442c-8692-3ec71caffa8b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2v285"
Sep 11 11:48:18.652418 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.651533    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2dd13752-864e-44e6-a1a5-986e623fbd7a-system-cni-dir\") pod \"multus-lh5dd\" (UID: \"2dd13752-864e-44e6-a1a5-986e623fbd7a\") " pod="openshift-multus/multus-lh5dd"
Sep 11 11:48:18.652418 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.651830    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cdaded97-b0ac-4688-8721-813fc905c103-tmp-dir\") pod \"node-resolver-2jqbl\" (UID: \"cdaded97-b0ac-4688-8721-813fc905c103\") " pod="openshift-dns/node-resolver-2jqbl"
Sep 11 11:48:18.652418 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.651835    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8fc69afd-c7cf-442c-8692-3ec71caffa8b-env-overrides\") pod \"ovnkube-node-2v285\" (UID: \"8fc69afd-c7cf-442c-8692-3ec71caffa8b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2v285"
Sep 11 11:48:18.652418 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.651886    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2dd13752-864e-44e6-a1a5-986e623fbd7a-host-run-k8s-cni-cncf-io\") pod \"multus-lh5dd\" (UID: \"2dd13752-864e-44e6-a1a5-986e623fbd7a\") " pod="openshift-multus/multus-lh5dd"
Sep 11 11:48:18.652418 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.651964    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/43cde4a1-c7f9-443d-9f2f-0ab809b5d25a-konnectivity-ca\") pod \"konnectivity-agent-dxf9l\" (UID: \"43cde4a1-c7f9-443d-9f2f-0ab809b5d25a\") " pod="kube-system/konnectivity-agent-dxf9l"
Sep 11 11:48:18.654753 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.654724    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/43cde4a1-c7f9-443d-9f2f-0ab809b5d25a-agent-certs\") pod \"konnectivity-agent-dxf9l\" (UID: \"43cde4a1-c7f9-443d-9f2f-0ab809b5d25a\") " pod="kube-system/konnectivity-agent-dxf9l"
Sep 11 11:48:18.655030 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.655008    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8fc69afd-c7cf-442c-8692-3ec71caffa8b-ovn-node-metrics-cert\") pod \"ovnkube-node-2v285\" (UID: \"8fc69afd-c7cf-442c-8692-3ec71caffa8b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2v285"
Sep 11 11:48:18.663742 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.663673    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sllf7\" (UniqueName: \"kubernetes.io/projected/cdaded97-b0ac-4688-8721-813fc905c103-kube-api-access-sllf7\") pod \"node-resolver-2jqbl\" (UID: \"cdaded97-b0ac-4688-8721-813fc905c103\") " pod="openshift-dns/node-resolver-2jqbl"
Sep 11 11:48:18.666702 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.666677    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zp4rk\" (UniqueName: \"kubernetes.io/projected/2dd13752-864e-44e6-a1a5-986e623fbd7a-kube-api-access-zp4rk\") pod \"multus-lh5dd\" (UID: \"2dd13752-864e-44e6-a1a5-986e623fbd7a\") " pod="openshift-multus/multus-lh5dd"
Sep 11 11:48:18.667665 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.667640    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-khcxv\" (UniqueName: \"kubernetes.io/projected/8fc69afd-c7cf-442c-8692-3ec71caffa8b-kube-api-access-khcxv\") pod \"ovnkube-node-2v285\" (UID: \"8fc69afd-c7cf-442c-8692-3ec71caffa8b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2v285"
Sep 11 11:48:18.744135 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.744097    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kbzcr"
Sep 11 11:48:18.753687 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.753654    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-vrgj4"
Sep 11 11:48:18.755403 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:18.755368    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcc619b6_bfb8_4cce_bf0c_b714377c7598.slice/crio-abb76ee814c652b889b7ca3dfb25acee5b42889c75569f3a53cac050e1fdb5dc WatchSource:0}: Error finding container abb76ee814c652b889b7ca3dfb25acee5b42889c75569f3a53cac050e1fdb5dc: Status 404 returned error can't find the container with id abb76ee814c652b889b7ca3dfb25acee5b42889c75569f3a53cac050e1fdb5dc
Sep 11 11:48:18.764065 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:18.764034    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fbe0c28_1c37_4360_b183_dd216ce48906.slice/crio-2492ae17900eb275b02fb787b7e15185e3f43954aafb2570f9a193f2256c16c9 WatchSource:0}: Error finding container 2492ae17900eb275b02fb787b7e15185e3f43954aafb2570f9a193f2256c16c9: Status 404 returned error can't find the container with id 2492ae17900eb275b02fb787b7e15185e3f43954aafb2570f9a193f2256c16c9
Sep 11 11:48:18.765308 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.765283    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-t7cd7"
Sep 11 11:48:18.773579 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.773543    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-pc2js"
Sep 11 11:48:18.778209 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:18.778179    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f52f2ea_a695_422d_a16c_fa3db1f93041.slice/crio-cbfb448cbde29343795df50a59aaf7f130f751853b80f9dec7edc5f404449bcf WatchSource:0}: Error finding container cbfb448cbde29343795df50a59aaf7f130f751853b80f9dec7edc5f404449bcf: Status 404 returned error can't find the container with id cbfb448cbde29343795df50a59aaf7f130f751853b80f9dec7edc5f404449bcf
Sep 11 11:48:18.783125 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:18.783096    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f1597c3_cf23_4523_bb1b_9003cc6f4b2a.slice/crio-c41f6b64bf8ef51b55e553efc3092d2280ec3f520d15880da0d9b2d9059b6efd WatchSource:0}: Error finding container c41f6b64bf8ef51b55e553efc3092d2280ec3f520d15880da0d9b2d9059b6efd: Status 404 returned error can't find the container with id c41f6b64bf8ef51b55e553efc3092d2280ec3f520d15880da0d9b2d9059b6efd
Sep 11 11:48:18.783233 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.783205    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4jthr"
Sep 11 11:48:18.793760 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.793735    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-dxf9l"
Sep 11 11:48:18.794167 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:18.794137    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7872368e_69ce_4a5c_9773_e2d02b8f06ba.slice/crio-10bbc39df7786c35c7c71dc54e9f0147741b0a78a28b838438d22d6b6f7ba5ab WatchSource:0}: Error finding container 10bbc39df7786c35c7c71dc54e9f0147741b0a78a28b838438d22d6b6f7ba5ab: Status 404 returned error can't find the container with id 10bbc39df7786c35c7c71dc54e9f0147741b0a78a28b838438d22d6b6f7ba5ab
Sep 11 11:48:18.804336 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.804310    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-2jqbl"
Sep 11 11:48:18.804728 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:18.804701    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43cde4a1_c7f9_443d_9f2f_0ab809b5d25a.slice/crio-6ce970e02ddbe77e48235b80512ddb0bb363b0d9e05a907de39377c3f640ee15 WatchSource:0}: Error finding container 6ce970e02ddbe77e48235b80512ddb0bb363b0d9e05a907de39377c3f640ee15: Status 404 returned error can't find the container with id 6ce970e02ddbe77e48235b80512ddb0bb363b0d9e05a907de39377c3f640ee15
Sep 11 11:48:18.812192 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.812165    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2v285"
Sep 11 11:48:18.815682 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:18.815650    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdaded97_b0ac_4688_8721_813fc905c103.slice/crio-87b634f56b924833172cc5fe23e34b76a938dbc5092c006177abb57cf1772845 WatchSource:0}: Error finding container 87b634f56b924833172cc5fe23e34b76a938dbc5092c006177abb57cf1772845: Status 404 returned error can't find the container with id 87b634f56b924833172cc5fe23e34b76a938dbc5092c006177abb57cf1772845
Sep 11 11:48:18.820219 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.820196    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lh5dd"
Sep 11 11:48:18.822130 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:18.822102    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fc69afd_c7cf_442c_8692_3ec71caffa8b.slice/crio-2e1b3f424a4a844f12560567ef2d39582a2743cc5799e567fe77e4badfb6a7ab WatchSource:0}: Error finding container 2e1b3f424a4a844f12560567ef2d39582a2743cc5799e567fe77e4badfb6a7ab: Status 404 returned error can't find the container with id 2e1b3f424a4a844f12560567ef2d39582a2743cc5799e567fe77e4badfb6a7ab
Sep 11 11:48:18.830257 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:18.830214    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2dd13752_864e_44e6_a1a5_986e623fbd7a.slice/crio-a4d2044bc271d015928318cea0a215863d7817752cdd2d1c917c6e80568e2fab WatchSource:0}: Error finding container a4d2044bc271d015928318cea0a215863d7817752cdd2d1c917c6e80568e2fab: Status 404 returned error can't find the container with id a4d2044bc271d015928318cea0a215863d7817752cdd2d1c917c6e80568e2fab
Sep 11 11:48:18.949855 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:18.949772    2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 11:48:19.053832 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:19.053790    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa7be6d8-e93e-49af-8782-14c8080fef33-metrics-certs\") pod \"network-metrics-daemon-krxsv\" (UID: \"fa7be6d8-e93e-49af-8782-14c8080fef33\") " pod="openshift-multus/network-metrics-daemon-krxsv"
Sep 11 11:48:19.054029 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:19.053943    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 11:48:19.054085 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:19.054033    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa7be6d8-e93e-49af-8782-14c8080fef33-metrics-certs podName:fa7be6d8-e93e-49af-8782-14c8080fef33 nodeName:}" failed. No retries permitted until 2026-09-11 11:48:20.054006304 +0000 UTC m=+3.951448031 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fa7be6d8-e93e-49af-8782-14c8080fef33-metrics-certs") pod "network-metrics-daemon-krxsv" (UID: "fa7be6d8-e93e-49af-8782-14c8080fef33") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 11:48:19.154571 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:19.154527    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/70994cc4-c221-4ab7-99ec-aed7c86955de-original-pull-secret\") pod \"global-pull-secret-syncer-79ltx\" (UID: \"70994cc4-c221-4ab7-99ec-aed7c86955de\") " pod="kube-system/global-pull-secret-syncer-79ltx"
Sep 11 11:48:19.154571 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:19.154572    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qbg9\" (UniqueName: \"kubernetes.io/projected/711f1752-03c4-4f9a-bb1b-cb7bf8cac9cd-kube-api-access-5qbg9\") pod \"network-check-target-dqc42\" (UID: \"711f1752-03c4-4f9a-bb1b-cb7bf8cac9cd\") " pod="openshift-network-diagnostics/network-check-target-dqc42"
Sep 11 11:48:19.154796 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:19.154724    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 11:48:19.154796 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:19.154725    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 11:48:19.154894 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:19.154821    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70994cc4-c221-4ab7-99ec-aed7c86955de-original-pull-secret podName:70994cc4-c221-4ab7-99ec-aed7c86955de nodeName:}" failed. No retries permitted until 2026-09-11 11:48:20.154793543 +0000 UTC m=+4.052235286 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/70994cc4-c221-4ab7-99ec-aed7c86955de-original-pull-secret") pod "global-pull-secret-syncer-79ltx" (UID: "70994cc4-c221-4ab7-99ec-aed7c86955de") : object "kube-system"/"original-pull-secret" not registered
Sep 11 11:48:19.154894 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:19.154742    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 11:48:19.154894 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:19.154854    2579 projected.go:196] Error preparing data for projected volume kube-api-access-5qbg9 for pod openshift-network-diagnostics/network-check-target-dqc42: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 11:48:19.155052 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:19.154907    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/711f1752-03c4-4f9a-bb1b-cb7bf8cac9cd-kube-api-access-5qbg9 podName:711f1752-03c4-4f9a-bb1b-cb7bf8cac9cd nodeName:}" failed. No retries permitted until 2026-09-11 11:48:20.154892691 +0000 UTC m=+4.052334416 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-5qbg9" (UniqueName: "kubernetes.io/projected/711f1752-03c4-4f9a-bb1b-cb7bf8cac9cd-kube-api-access-5qbg9") pod "network-check-target-dqc42" (UID: "711f1752-03c4-4f9a-bb1b-cb7bf8cac9cd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 11:48:19.486220 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:19.486139    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 11:43:18 +0000 UTC" deadline="2028-02-05 16:31:26.889316474 +0000 UTC"
Sep 11 11:48:19.486220 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:19.486216    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12292h43m7.403105307s"
Sep 11 11:48:19.569849 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:19.569812    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dqc42"
Sep 11 11:48:19.570038 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:19.569945    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dqc42" podUID="711f1752-03c4-4f9a-bb1b-cb7bf8cac9cd"
Sep 11 11:48:19.570038 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:19.570019    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79ltx"
Sep 11 11:48:19.570162 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:19.570140    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79ltx" podUID="70994cc4-c221-4ab7-99ec-aed7c86955de"
Sep 11 11:48:19.581989 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:19.581953    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-dxf9l" event={"ID":"43cde4a1-c7f9-443d-9f2f-0ab809b5d25a","Type":"ContainerStarted","Data":"6ce970e02ddbe77e48235b80512ddb0bb363b0d9e05a907de39377c3f640ee15"}
Sep 11 11:48:19.583294 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:19.583268    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4jthr" event={"ID":"7872368e-69ce-4a5c-9773-e2d02b8f06ba","Type":"ContainerStarted","Data":"10bbc39df7786c35c7c71dc54e9f0147741b0a78a28b838438d22d6b6f7ba5ab"}
Sep 11 11:48:19.584678 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:19.584654    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-pc2js" event={"ID":"5f1597c3-cf23-4523-bb1b-9003cc6f4b2a","Type":"ContainerStarted","Data":"c41f6b64bf8ef51b55e553efc3092d2280ec3f520d15880da0d9b2d9059b6efd"}
Sep 11 11:48:19.585989 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:19.585965    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-vrgj4" event={"ID":"9fbe0c28-1c37-4360-b183-dd216ce48906","Type":"ContainerStarted","Data":"2492ae17900eb275b02fb787b7e15185e3f43954aafb2570f9a193f2256c16c9"}
Sep 11 11:48:19.587356 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:19.587317    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kbzcr" event={"ID":"bcc619b6-bfb8-4cce-bf0c-b714377c7598","Type":"ContainerStarted","Data":"abb76ee814c652b889b7ca3dfb25acee5b42889c75569f3a53cac050e1fdb5dc"}
Sep 11 11:48:19.588695 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:19.588673    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lh5dd" event={"ID":"2dd13752-864e-44e6-a1a5-986e623fbd7a","Type":"ContainerStarted","Data":"a4d2044bc271d015928318cea0a215863d7817752cdd2d1c917c6e80568e2fab"}
Sep 11 11:48:19.590001 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:19.589978    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2v285" event={"ID":"8fc69afd-c7cf-442c-8692-3ec71caffa8b","Type":"ContainerStarted","Data":"2e1b3f424a4a844f12560567ef2d39582a2743cc5799e567fe77e4badfb6a7ab"}
Sep 11 11:48:19.592711 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:19.592685    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-2jqbl" event={"ID":"cdaded97-b0ac-4688-8721-813fc905c103","Type":"ContainerStarted","Data":"87b634f56b924833172cc5fe23e34b76a938dbc5092c006177abb57cf1772845"}
Sep 11 11:48:19.593975 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:19.593952    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-t7cd7" event={"ID":"0f52f2ea-a695-422d-a16c-fa3db1f93041","Type":"ContainerStarted","Data":"cbfb448cbde29343795df50a59aaf7f130f751853b80f9dec7edc5f404449bcf"}
Sep 11 11:48:20.064077 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:20.064021    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa7be6d8-e93e-49af-8782-14c8080fef33-metrics-certs\") pod \"network-metrics-daemon-krxsv\" (UID: \"fa7be6d8-e93e-49af-8782-14c8080fef33\") " pod="openshift-multus/network-metrics-daemon-krxsv"
Sep 11 11:48:20.064284 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:20.064260    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 11:48:20.064352 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:20.064324    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa7be6d8-e93e-49af-8782-14c8080fef33-metrics-certs podName:fa7be6d8-e93e-49af-8782-14c8080fef33 nodeName:}" failed. No retries permitted until 2026-09-11 11:48:22.064305809 +0000 UTC m=+5.961747539 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fa7be6d8-e93e-49af-8782-14c8080fef33-metrics-certs") pod "network-metrics-daemon-krxsv" (UID: "fa7be6d8-e93e-49af-8782-14c8080fef33") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 11:48:20.165392 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:20.165348    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/70994cc4-c221-4ab7-99ec-aed7c86955de-original-pull-secret\") pod \"global-pull-secret-syncer-79ltx\" (UID: \"70994cc4-c221-4ab7-99ec-aed7c86955de\") " pod="kube-system/global-pull-secret-syncer-79ltx"
Sep 11 11:48:20.165661 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:20.165408    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qbg9\" (UniqueName: \"kubernetes.io/projected/711f1752-03c4-4f9a-bb1b-cb7bf8cac9cd-kube-api-access-5qbg9\") pod \"network-check-target-dqc42\" (UID: \"711f1752-03c4-4f9a-bb1b-cb7bf8cac9cd\") " pod="openshift-network-diagnostics/network-check-target-dqc42"
Sep 11 11:48:20.165661 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:20.165541    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 11:48:20.165661 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:20.165609    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 11:48:20.165661 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:20.165629    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 11:48:20.165661 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:20.165643    2579 projected.go:196] Error preparing data for projected volume kube-api-access-5qbg9 for pod openshift-network-diagnostics/network-check-target-dqc42: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 11:48:20.165930 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:20.165711    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70994cc4-c221-4ab7-99ec-aed7c86955de-original-pull-secret podName:70994cc4-c221-4ab7-99ec-aed7c86955de nodeName:}" failed. No retries permitted until 2026-09-11 11:48:22.165612239 +0000 UTC m=+6.063053968 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/70994cc4-c221-4ab7-99ec-aed7c86955de-original-pull-secret") pod "global-pull-secret-syncer-79ltx" (UID: "70994cc4-c221-4ab7-99ec-aed7c86955de") : object "kube-system"/"original-pull-secret" not registered
Sep 11 11:48:20.165930 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:20.165744    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/711f1752-03c4-4f9a-bb1b-cb7bf8cac9cd-kube-api-access-5qbg9 podName:711f1752-03c4-4f9a-bb1b-cb7bf8cac9cd nodeName:}" failed. No retries permitted until 2026-09-11 11:48:22.165732237 +0000 UTC m=+6.063173965 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-5qbg9" (UniqueName: "kubernetes.io/projected/711f1752-03c4-4f9a-bb1b-cb7bf8cac9cd-kube-api-access-5qbg9") pod "network-check-target-dqc42" (UID: "711f1752-03c4-4f9a-bb1b-cb7bf8cac9cd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 11:48:20.487348 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:20.487252    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 11:43:18 +0000 UTC" deadline="2028-04-23 16:27:53.440558736 +0000 UTC"
Sep 11 11:48:20.487348 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:20.487296    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14164h39m32.953268404s"
Sep 11 11:48:20.574869 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:20.574036    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-krxsv"
Sep 11 11:48:20.574869 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:20.574201    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-krxsv" podUID="fa7be6d8-e93e-49af-8782-14c8080fef33"
Sep 11 11:48:20.613815 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:20.612701    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-145.ec2.internal" event={"ID":"3d9525f1a3218f8a340d89c0b17b6cdb","Type":"ContainerStarted","Data":"16957a656ab5fc1bf46f3886cf7a462c44fdf05421be11b5416d4cd6c70701eb"}
Sep 11 11:48:21.570358 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:21.570324    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79ltx"
Sep 11 11:48:21.570358 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:21.570349    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dqc42"
Sep 11 11:48:21.571041 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:21.570488    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79ltx" podUID="70994cc4-c221-4ab7-99ec-aed7c86955de"
Sep 11 11:48:21.571041 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:21.570596    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dqc42" podUID="711f1752-03c4-4f9a-bb1b-cb7bf8cac9cd"
Sep 11 11:48:21.625490 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:21.624830    2579 generic.go:356] "Generic (PLEG): container finished" podID="0a88adf83eef5bb51fa329190cac6245" containerID="e1f3991f3773f9e9a5540bdac5907cc1adaaa09b411daa2c50ae714b321bf7bd" exitCode=0
Sep 11 11:48:21.625490 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:21.624947    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-145.ec2.internal" event={"ID":"0a88adf83eef5bb51fa329190cac6245","Type":"ContainerDied","Data":"e1f3991f3773f9e9a5540bdac5907cc1adaaa09b411daa2c50ae714b321bf7bd"}
Sep 11 11:48:21.643551 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:21.642556    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-145.ec2.internal" podStartSLOduration=4.642532497 podStartE2EDuration="4.642532497s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 11:48:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:48:20.66254714 +0000 UTC m=+4.559988891" watchObservedRunningTime="2026-09-11 11:48:21.642532497 +0000 UTC m=+5.539974243"
Sep 11 11:48:22.086688 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:22.086254    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa7be6d8-e93e-49af-8782-14c8080fef33-metrics-certs\") pod \"network-metrics-daemon-krxsv\" (UID: \"fa7be6d8-e93e-49af-8782-14c8080fef33\") " pod="openshift-multus/network-metrics-daemon-krxsv"
Sep 11 11:48:22.086688 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:22.086540    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 11:48:22.086688 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:22.086637    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa7be6d8-e93e-49af-8782-14c8080fef33-metrics-certs podName:fa7be6d8-e93e-49af-8782-14c8080fef33 nodeName:}" failed. No retries permitted until 2026-09-11 11:48:26.086616381 +0000 UTC m=+9.984058119 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fa7be6d8-e93e-49af-8782-14c8080fef33-metrics-certs") pod "network-metrics-daemon-krxsv" (UID: "fa7be6d8-e93e-49af-8782-14c8080fef33") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 11:48:22.187771 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:22.187683    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/70994cc4-c221-4ab7-99ec-aed7c86955de-original-pull-secret\") pod \"global-pull-secret-syncer-79ltx\" (UID: \"70994cc4-c221-4ab7-99ec-aed7c86955de\") " pod="kube-system/global-pull-secret-syncer-79ltx"
Sep 11 11:48:22.187771 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:22.187739    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qbg9\" (UniqueName: \"kubernetes.io/projected/711f1752-03c4-4f9a-bb1b-cb7bf8cac9cd-kube-api-access-5qbg9\") pod \"network-check-target-dqc42\" (UID: \"711f1752-03c4-4f9a-bb1b-cb7bf8cac9cd\") " pod="openshift-network-diagnostics/network-check-target-dqc42"
Sep 11 11:48:22.188004 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:22.187891    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 11:48:22.188004 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:22.187918    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 11:48:22.188004 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:22.187930    2579 projected.go:196] Error preparing data for projected volume kube-api-access-5qbg9 for pod openshift-network-diagnostics/network-check-target-dqc42: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 11:48:22.188004 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:22.187992    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/711f1752-03c4-4f9a-bb1b-cb7bf8cac9cd-kube-api-access-5qbg9 podName:711f1752-03c4-4f9a-bb1b-cb7bf8cac9cd nodeName:}" failed. No retries permitted until 2026-09-11 11:48:26.187972245 +0000 UTC m=+10.085413988 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-5qbg9" (UniqueName: "kubernetes.io/projected/711f1752-03c4-4f9a-bb1b-cb7bf8cac9cd-kube-api-access-5qbg9") pod "network-check-target-dqc42" (UID: "711f1752-03c4-4f9a-bb1b-cb7bf8cac9cd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 11:48:22.188437 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:22.188412    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 11:48:22.188527 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:22.188477    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70994cc4-c221-4ab7-99ec-aed7c86955de-original-pull-secret podName:70994cc4-c221-4ab7-99ec-aed7c86955de nodeName:}" failed. No retries permitted until 2026-09-11 11:48:26.188459135 +0000 UTC m=+10.085900862 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/70994cc4-c221-4ab7-99ec-aed7c86955de-original-pull-secret") pod "global-pull-secret-syncer-79ltx" (UID: "70994cc4-c221-4ab7-99ec-aed7c86955de") : object "kube-system"/"original-pull-secret" not registered
Sep 11 11:48:22.573024 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:22.572947    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-krxsv"
Sep 11 11:48:22.573488 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:22.573092    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-krxsv" podUID="fa7be6d8-e93e-49af-8782-14c8080fef33"
Sep 11 11:48:22.632643 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:22.632360    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-145.ec2.internal" event={"ID":"0a88adf83eef5bb51fa329190cac6245","Type":"ContainerStarted","Data":"002e58e1ef899ebdf998053199d446e285203380196721e6f3d7733f77c9f1cb"}
Sep 11 11:48:23.570021 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:23.569985    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79ltx"
Sep 11 11:48:23.570217 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:23.570133    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79ltx" podUID="70994cc4-c221-4ab7-99ec-aed7c86955de"
Sep 11 11:48:23.570217 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:23.570169    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dqc42"
Sep 11 11:48:23.570329 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:23.570285    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dqc42" podUID="711f1752-03c4-4f9a-bb1b-cb7bf8cac9cd"
Sep 11 11:48:24.570953 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:24.570886    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-krxsv"
Sep 11 11:48:24.571487 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:24.571013    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-krxsv" podUID="fa7be6d8-e93e-49af-8782-14c8080fef33"
Sep 11 11:48:25.569681 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:25.569602    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79ltx"
Sep 11 11:48:25.569681 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:25.569642    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dqc42"
Sep 11 11:48:25.569917 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:25.569746    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79ltx" podUID="70994cc4-c221-4ab7-99ec-aed7c86955de"
Sep 11 11:48:25.569917 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:25.569885    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dqc42" podUID="711f1752-03c4-4f9a-bb1b-cb7bf8cac9cd"
Sep 11 11:48:26.122929 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:26.122893    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa7be6d8-e93e-49af-8782-14c8080fef33-metrics-certs\") pod \"network-metrics-daemon-krxsv\" (UID: \"fa7be6d8-e93e-49af-8782-14c8080fef33\") " pod="openshift-multus/network-metrics-daemon-krxsv"
Sep 11 11:48:26.123296 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:26.123071    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 11:48:26.123296 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:26.123150    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa7be6d8-e93e-49af-8782-14c8080fef33-metrics-certs podName:fa7be6d8-e93e-49af-8782-14c8080fef33 nodeName:}" failed. No retries permitted until 2026-09-11 11:48:34.123128243 +0000 UTC m=+18.020569972 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fa7be6d8-e93e-49af-8782-14c8080fef33-metrics-certs") pod "network-metrics-daemon-krxsv" (UID: "fa7be6d8-e93e-49af-8782-14c8080fef33") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 11:48:26.224280 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:26.224239    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qbg9\" (UniqueName: \"kubernetes.io/projected/711f1752-03c4-4f9a-bb1b-cb7bf8cac9cd-kube-api-access-5qbg9\") pod \"network-check-target-dqc42\" (UID: \"711f1752-03c4-4f9a-bb1b-cb7bf8cac9cd\") " pod="openshift-network-diagnostics/network-check-target-dqc42"
Sep 11 11:48:26.224475 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:26.224352    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/70994cc4-c221-4ab7-99ec-aed7c86955de-original-pull-secret\") pod \"global-pull-secret-syncer-79ltx\" (UID: \"70994cc4-c221-4ab7-99ec-aed7c86955de\") " pod="kube-system/global-pull-secret-syncer-79ltx"
Sep 11 11:48:26.224634 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:26.224479    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 11:48:26.224634 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:26.224499    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 11:48:26.224634 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:26.224532    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 11:48:26.224634 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:26.224547    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70994cc4-c221-4ab7-99ec-aed7c86955de-original-pull-secret podName:70994cc4-c221-4ab7-99ec-aed7c86955de nodeName:}" failed. No retries permitted until 2026-09-11 11:48:34.224525369 +0000 UTC m=+18.121967115 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/70994cc4-c221-4ab7-99ec-aed7c86955de-original-pull-secret") pod "global-pull-secret-syncer-79ltx" (UID: "70994cc4-c221-4ab7-99ec-aed7c86955de") : object "kube-system"/"original-pull-secret" not registered
Sep 11 11:48:26.224634 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:26.224563    2579 projected.go:196] Error preparing data for projected volume kube-api-access-5qbg9 for pod openshift-network-diagnostics/network-check-target-dqc42: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 11:48:26.224634 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:26.224622    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/711f1752-03c4-4f9a-bb1b-cb7bf8cac9cd-kube-api-access-5qbg9 podName:711f1752-03c4-4f9a-bb1b-cb7bf8cac9cd nodeName:}" failed. No retries permitted until 2026-09-11 11:48:34.224604439 +0000 UTC m=+18.122046177 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-5qbg9" (UniqueName: "kubernetes.io/projected/711f1752-03c4-4f9a-bb1b-cb7bf8cac9cd-kube-api-access-5qbg9") pod "network-check-target-dqc42" (UID: "711f1752-03c4-4f9a-bb1b-cb7bf8cac9cd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 11:48:26.572044 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:26.571522    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-krxsv"
Sep 11 11:48:26.572044 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:26.571656    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-krxsv" podUID="fa7be6d8-e93e-49af-8782-14c8080fef33"
Sep 11 11:48:27.570558 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:27.570520    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dqc42"
Sep 11 11:48:27.570994 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:27.570579    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79ltx"
Sep 11 11:48:27.570994 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:27.570663    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dqc42" podUID="711f1752-03c4-4f9a-bb1b-cb7bf8cac9cd"
Sep 11 11:48:27.570994 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:27.570774    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79ltx" podUID="70994cc4-c221-4ab7-99ec-aed7c86955de"
Sep 11 11:48:28.570029 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:28.569975    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-krxsv"
Sep 11 11:48:28.570213 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:28.570127    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-krxsv" podUID="fa7be6d8-e93e-49af-8782-14c8080fef33"
Sep 11 11:48:29.571591 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:29.570140    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79ltx"
Sep 11 11:48:29.571591 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:29.570188    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dqc42"
Sep 11 11:48:29.571591 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:29.570270    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79ltx" podUID="70994cc4-c221-4ab7-99ec-aed7c86955de"
Sep 11 11:48:29.571591 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:29.570386    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dqc42" podUID="711f1752-03c4-4f9a-bb1b-cb7bf8cac9cd"
Sep 11 11:48:30.573160 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:30.573127    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-krxsv"
Sep 11 11:48:30.573598 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:30.573261    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-krxsv" podUID="fa7be6d8-e93e-49af-8782-14c8080fef33"
Sep 11 11:48:31.570102 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:31.570059    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dqc42"
Sep 11 11:48:31.570102 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:31.570097    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79ltx"
Sep 11 11:48:31.570368 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:31.570205    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dqc42" podUID="711f1752-03c4-4f9a-bb1b-cb7bf8cac9cd"
Sep 11 11:48:31.570368 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:31.570331    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79ltx" podUID="70994cc4-c221-4ab7-99ec-aed7c86955de"
Sep 11 11:48:32.570696 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:32.570662    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-krxsv"
Sep 11 11:48:32.571177 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:32.570808    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-krxsv" podUID="fa7be6d8-e93e-49af-8782-14c8080fef33"
Sep 11 11:48:33.569757 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:33.569719    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79ltx"
Sep 11 11:48:33.569956 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:33.569826    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79ltx" podUID="70994cc4-c221-4ab7-99ec-aed7c86955de"
Sep 11 11:48:33.569956 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:33.569884    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dqc42"
Sep 11 11:48:33.570074 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:33.570009    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dqc42" podUID="711f1752-03c4-4f9a-bb1b-cb7bf8cac9cd"
Sep 11 11:48:34.179096 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:34.179060    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa7be6d8-e93e-49af-8782-14c8080fef33-metrics-certs\") pod \"network-metrics-daemon-krxsv\" (UID: \"fa7be6d8-e93e-49af-8782-14c8080fef33\") " pod="openshift-multus/network-metrics-daemon-krxsv"
Sep 11 11:48:34.179583 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:34.179186    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 11:48:34.179583 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:34.179240    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa7be6d8-e93e-49af-8782-14c8080fef33-metrics-certs podName:fa7be6d8-e93e-49af-8782-14c8080fef33 nodeName:}" failed. No retries permitted until 2026-09-11 11:48:50.179225827 +0000 UTC m=+34.076667564 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fa7be6d8-e93e-49af-8782-14c8080fef33-metrics-certs") pod "network-metrics-daemon-krxsv" (UID: "fa7be6d8-e93e-49af-8782-14c8080fef33") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 11:48:34.279750 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:34.279712    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/70994cc4-c221-4ab7-99ec-aed7c86955de-original-pull-secret\") pod \"global-pull-secret-syncer-79ltx\" (UID: \"70994cc4-c221-4ab7-99ec-aed7c86955de\") " pod="kube-system/global-pull-secret-syncer-79ltx"
Sep 11 11:48:34.279750 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:34.279756    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qbg9\" (UniqueName: \"kubernetes.io/projected/711f1752-03c4-4f9a-bb1b-cb7bf8cac9cd-kube-api-access-5qbg9\") pod \"network-check-target-dqc42\" (UID: \"711f1752-03c4-4f9a-bb1b-cb7bf8cac9cd\") " pod="openshift-network-diagnostics/network-check-target-dqc42"
Sep 11 11:48:34.279977 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:34.279862    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 11:48:34.279977 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:34.279876    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 11:48:34.279977 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:34.279889    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 11:48:34.279977 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:34.279901    2579 projected.go:196] Error preparing data for projected volume kube-api-access-5qbg9 for pod openshift-network-diagnostics/network-check-target-dqc42: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 11:48:34.279977 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:34.279932    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70994cc4-c221-4ab7-99ec-aed7c86955de-original-pull-secret podName:70994cc4-c221-4ab7-99ec-aed7c86955de nodeName:}" failed. No retries permitted until 2026-09-11 11:48:50.279910629 +0000 UTC m=+34.177352365 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/70994cc4-c221-4ab7-99ec-aed7c86955de-original-pull-secret") pod "global-pull-secret-syncer-79ltx" (UID: "70994cc4-c221-4ab7-99ec-aed7c86955de") : object "kube-system"/"original-pull-secret" not registered
Sep 11 11:48:34.279977 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:34.279952    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/711f1752-03c4-4f9a-bb1b-cb7bf8cac9cd-kube-api-access-5qbg9 podName:711f1752-03c4-4f9a-bb1b-cb7bf8cac9cd nodeName:}" failed. No retries permitted until 2026-09-11 11:48:50.279942576 +0000 UTC m=+34.177384300 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-5qbg9" (UniqueName: "kubernetes.io/projected/711f1752-03c4-4f9a-bb1b-cb7bf8cac9cd-kube-api-access-5qbg9") pod "network-check-target-dqc42" (UID: "711f1752-03c4-4f9a-bb1b-cb7bf8cac9cd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 11:48:34.570529 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:34.570447    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-krxsv"
Sep 11 11:48:34.570697 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:34.570603    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-krxsv" podUID="fa7be6d8-e93e-49af-8782-14c8080fef33"
Sep 11 11:48:35.570420 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:35.570371    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dqc42"
Sep 11 11:48:35.570908 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:35.570515    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dqc42" podUID="711f1752-03c4-4f9a-bb1b-cb7bf8cac9cd"
Sep 11 11:48:35.570908 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:35.570575    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79ltx"
Sep 11 11:48:35.570908 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:35.570675    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79ltx" podUID="70994cc4-c221-4ab7-99ec-aed7c86955de"
Sep 11 11:48:36.570700 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:36.570667    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-krxsv"
Sep 11 11:48:36.571123 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:36.570786    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-krxsv" podUID="fa7be6d8-e93e-49af-8782-14c8080fef33"
Sep 11 11:48:37.570369 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:37.570237    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dqc42"
Sep 11 11:48:37.570570 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:37.570520    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dqc42" podUID="711f1752-03c4-4f9a-bb1b-cb7bf8cac9cd"
Sep 11 11:48:37.570740 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:37.570608    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79ltx"
Sep 11 11:48:37.570740 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:37.570677    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79ltx" podUID="70994cc4-c221-4ab7-99ec-aed7c86955de"
Sep 11 11:48:37.665167 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:37.665137    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-dxf9l" event={"ID":"43cde4a1-c7f9-443d-9f2f-0ab809b5d25a","Type":"ContainerStarted","Data":"23001ecaa89fd0dd9e41d9a435edb1440e4bb7cfd280b9c1324e560bb3278720"}
Sep 11 11:48:37.667350 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:37.667316    2579 generic.go:356] "Generic (PLEG): container finished" podID="7872368e-69ce-4a5c-9773-e2d02b8f06ba" containerID="55e6822265a2fd416135fb5d65297f005b9054fdc3248ff54ae844cd0bc99de4" exitCode=0
Sep 11 11:48:37.667494 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:37.667433    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4jthr" event={"ID":"7872368e-69ce-4a5c-9773-e2d02b8f06ba","Type":"ContainerDied","Data":"55e6822265a2fd416135fb5d65297f005b9054fdc3248ff54ae844cd0bc99de4"}
Sep 11 11:48:37.669262 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:37.669231    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-vrgj4" event={"ID":"9fbe0c28-1c37-4360-b183-dd216ce48906","Type":"ContainerStarted","Data":"4e8bb5629a2051463b0308cb4bba4bdbaea374e6bf683d574f82ce0e7476ece7"}
Sep 11 11:48:37.671052 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:37.671032    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kbzcr" event={"ID":"bcc619b6-bfb8-4cce-bf0c-b714377c7598","Type":"ContainerStarted","Data":"8e97c88ef10026b23551fb4878f4a9790fc73254be9e44e3bd29258f5d7a836e"}
Sep 11 11:48:37.672839 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:37.672804    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lh5dd" event={"ID":"2dd13752-864e-44e6-a1a5-986e623fbd7a","Type":"ContainerStarted","Data":"a063b36cc9f584a748b9d588ce46f4117adf88454da6fe3415af1ae48aff5f10"}
Sep 11 11:48:37.675934 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:37.675878    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2v285" event={"ID":"8fc69afd-c7cf-442c-8692-3ec71caffa8b","Type":"ContainerStarted","Data":"617e244cc5f3d97351c8790454cf6e5184331834d83fb5640ff01ddd6f31154e"}
Sep 11 11:48:37.676039 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:37.676021    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2v285" event={"ID":"8fc69afd-c7cf-442c-8692-3ec71caffa8b","Type":"ContainerStarted","Data":"9669a9868e2ddf2bb6c4f12b27113b62272332b75e0b9ecfd1630138f61a484f"}
Sep 11 11:48:37.676119 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:37.676079    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2v285" event={"ID":"8fc69afd-c7cf-442c-8692-3ec71caffa8b","Type":"ContainerStarted","Data":"7b81bc6f8dedbedcc7c80104e862830bc73210974aebfed9c3e29b15d740d792"}
Sep 11 11:48:37.677650 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:37.677630    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-2jqbl" event={"ID":"cdaded97-b0ac-4688-8721-813fc905c103","Type":"ContainerStarted","Data":"e4dcb6fa77e86f53feb58d14c940aebeca70606d8ad6269a82b66f56a193499f"}
Sep 11 11:48:37.679171 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:37.679149    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-t7cd7" event={"ID":"0f52f2ea-a695-422d-a16c-fa3db1f93041","Type":"ContainerStarted","Data":"2451a89cf6fd9e64df023de58f05fd13b053bb431ae5cf6bd3faaa330171c7e1"}
Sep 11 11:48:37.681952 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:37.681900    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-dxf9l" podStartSLOduration=6.031575179 podStartE2EDuration="20.681888469s" totalImagesPullingTime="14.65031329s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:48:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:48:37.68174482 +0000 UTC m=+21.579186568" watchObservedRunningTime="2026-09-11 11:48:37.681888469 +0000 UTC m=+21.579330216"
Sep 11 11:48:37.682129 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:37.682056    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-145.ec2.internal" podStartSLOduration=20.682048351 podStartE2EDuration="20.682048351s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 11:48:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:48:22.650281936 +0000 UTC m=+6.547723682" watchObservedRunningTime="2026-09-11 11:48:37.682048351 +0000 UTC m=+21.579490104"
Sep 11 11:48:37.706235 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:37.703957    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-lh5dd" podStartSLOduration=2.1553556560000002 podStartE2EDuration="20.703934598s" totalImagesPullingTime="18.548578942s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 11:48:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:48:37.703671627 +0000 UTC m=+21.601113372" watchObservedRunningTime="2026-09-11 11:48:37.703934598 +0000 UTC m=+21.601376406"
Sep 11 11:48:37.728512 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:37.728478    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:48:37.741036 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:37.740984    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-vrgj4" podStartSLOduration=3.177101838 podStartE2EDuration="21.740962579s" totalImagesPullingTime="18.563860741s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 11:48:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:48:37.724717864 +0000 UTC m=+21.622159609" watchObservedRunningTime="2026-09-11 11:48:37.740962579 +0000 UTC m=+21.638404325"
Sep 11 11:48:37.741295 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:37.741127    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-t7cd7" podStartSLOduration=3.193162728 podStartE2EDuration="21.741119247s" totalImagesPullingTime="18.547956519s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 11:48:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:48:37.740957449 +0000 UTC m=+21.638399194" watchObservedRunningTime="2026-09-11 11:48:37.741119247 +0000 UTC m=+21.638560994"
Sep 11 11:48:37.783983 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:37.783939    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-2jqbl" podStartSLOduration=2.273330833 podStartE2EDuration="20.783923849s" totalImagesPullingTime="18.510593016s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 11:48:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:48:37.783410608 +0000 UTC m=+21.680852353" watchObservedRunningTime="2026-09-11 11:48:37.783923849 +0000 UTC m=+21.681365593"
Sep 11 11:48:37.829245 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:37.829201    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:48:37.929995 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:37.929757    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:48:38.018864 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:38.018828    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:48:38.309003 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:38.308881    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-dxf9l"
Sep 11 11:48:38.360359 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:38.360324    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-dxf9l"
Sep 11 11:48:38.361008 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:38.360989    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-dxf9l"
Sep 11 11:48:38.570283 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:38.570201    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-krxsv"
Sep 11 11:48:38.570506 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:38.570327    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-krxsv" podUID="fa7be6d8-e93e-49af-8782-14c8080fef33"
Sep 11 11:48:38.685093 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:38.685056    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2v285" event={"ID":"8fc69afd-c7cf-442c-8692-3ec71caffa8b","Type":"ContainerStarted","Data":"73f9820c1ccb73a88335e329ef4878331963f35326df3475db2a0db48ea8db8d"}
Sep 11 11:48:38.685747 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:38.685149    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2v285" event={"ID":"8fc69afd-c7cf-442c-8692-3ec71caffa8b","Type":"ContainerStarted","Data":"a31ab4fea835b6ba86f0e03b794e5611d49b46bbc7085c01e3e397018548850b"}
Sep 11 11:48:38.685747 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:38.685201    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2v285" event={"ID":"8fc69afd-c7cf-442c-8692-3ec71caffa8b","Type":"ContainerStarted","Data":"21072fbf8a06f660d1a30ebbeb5dd905a5b4cacdac6f5e0956ccd276fee71889"}
Sep 11 11:48:38.685912 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:38.685893    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-dxf9l"
Sep 11 11:48:38.769821 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:38.769790    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:48:39.055582 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:39.055557    2579 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 11 11:48:39.494778 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:39.494671    2579 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-11T11:48:39.055575011Z","UUID":"42fd2016-ea80-4e6b-937d-7e43f27c8ee1","Handler":null,"Name":"","Endpoint":""}
Sep 11 11:48:39.496282 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:39.496257    2579 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 11 11:48:39.496422 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:39.496289    2579 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 11 11:48:39.570566 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:39.570523    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dqc42"
Sep 11 11:48:39.570724 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:39.570523    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79ltx"
Sep 11 11:48:39.570724 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:39.570667    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dqc42" podUID="711f1752-03c4-4f9a-bb1b-cb7bf8cac9cd"
Sep 11 11:48:39.570724 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:39.570701    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79ltx" podUID="70994cc4-c221-4ab7-99ec-aed7c86955de"
Sep 11 11:48:39.689457 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:39.689280    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-pc2js" event={"ID":"5f1597c3-cf23-4523-bb1b-9003cc6f4b2a","Type":"ContainerStarted","Data":"f645d36558461800f25e795189efdd4d53d66819e01af8b2e7b8676350ac337e"}
Sep 11 11:48:39.691456 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:39.691431    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kbzcr" event={"ID":"bcc619b6-bfb8-4cce-bf0c-b714377c7598","Type":"ContainerStarted","Data":"de230f10da07055af5925d9ff50ddce45528bb4f2e3549d34ab8f472bae25586"}
Sep 11 11:48:39.708681 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:39.708624    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-pc2js" podStartSLOduration=5.165290042 podStartE2EDuration="23.708606837s" totalImagesPullingTime="18.543316795s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 11:48:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:48:39.708095013 +0000 UTC m=+23.605536759" watchObservedRunningTime="2026-09-11 11:48:39.708606837 +0000 UTC m=+23.606048582"
Sep 11 11:48:39.834159 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:39.834021    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:48:39.935568 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:39.935528    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:48:40.569637 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:40.569606    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-krxsv"
Sep 11 11:48:40.569818 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:40.569726    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-krxsv" podUID="fa7be6d8-e93e-49af-8782-14c8080fef33"
Sep 11 11:48:40.697498 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:40.697449    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2v285" event={"ID":"8fc69afd-c7cf-442c-8692-3ec71caffa8b","Type":"ContainerStarted","Data":"6bafd0c600958094eee47b4f72509116e21aa198b381707b7ae1454a3fc146e2"}
Sep 11 11:48:41.570125 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:41.570093    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79ltx"
Sep 11 11:48:41.570296 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:41.570096    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dqc42"
Sep 11 11:48:41.570296 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:41.570212    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79ltx" podUID="70994cc4-c221-4ab7-99ec-aed7c86955de"
Sep 11 11:48:41.570384 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:41.570295    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dqc42" podUID="711f1752-03c4-4f9a-bb1b-cb7bf8cac9cd"
Sep 11 11:48:42.570532 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:42.570351    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-krxsv"
Sep 11 11:48:42.571090 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:42.570638    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-krxsv" podUID="fa7be6d8-e93e-49af-8782-14c8080fef33"
Sep 11 11:48:42.704110 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:42.704078    2579 generic.go:356] "Generic (PLEG): container finished" podID="7872368e-69ce-4a5c-9773-e2d02b8f06ba" containerID="4186a7ba00a9c94e1d2d6f463a98c5793be0a121faaf512268616ad7bf25d921" exitCode=0
Sep 11 11:48:42.704265 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:42.704171    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4jthr" event={"ID":"7872368e-69ce-4a5c-9773-e2d02b8f06ba","Type":"ContainerDied","Data":"4186a7ba00a9c94e1d2d6f463a98c5793be0a121faaf512268616ad7bf25d921"}
Sep 11 11:48:42.710049 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:42.710016    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kbzcr" event={"ID":"bcc619b6-bfb8-4cce-bf0c-b714377c7598","Type":"ContainerStarted","Data":"c6e991c386349cd1eda300a7761c1eda432c2344d42b5012092b5e603fa28aa6"}
Sep 11 11:48:42.713752 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:42.713732    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2v285" event={"ID":"8fc69afd-c7cf-442c-8692-3ec71caffa8b","Type":"ContainerStarted","Data":"78a3020c5ec640a34d33257ff6c620ca45d64dc01dcb1883d8d3c6351e1b3c3e"}
Sep 11 11:48:42.714067 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:42.714053    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2v285"
Sep 11 11:48:42.728957 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:42.728932    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2v285"
Sep 11 11:48:42.756644 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:42.756594    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-2v285" podStartSLOduration=7.202413833 podStartE2EDuration="25.756579183s" totalImagesPullingTime="18.55416535s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 11:48:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:48:42.755215129 +0000 UTC m=+26.652656873" watchObservedRunningTime="2026-09-11 11:48:42.756579183 +0000 UTC m=+26.654020985"
Sep 11 11:48:42.775222 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:42.775182    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kbzcr" podStartSLOduration=3.753632401 podStartE2EDuration="26.775168379s" totalImagesPullingTime="23.021535978s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 11:48:16 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:48:42.774969271 +0000 UTC m=+26.672411016" watchObservedRunningTime="2026-09-11 11:48:42.775168379 +0000 UTC m=+26.672610124"
Sep 11 11:48:43.570488 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:43.570410    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79ltx"
Sep 11 11:48:43.570778 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:43.570460    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dqc42"
Sep 11 11:48:43.571370 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:43.570798    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79ltx" podUID="70994cc4-c221-4ab7-99ec-aed7c86955de"
Sep 11 11:48:43.571370 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:43.570861    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dqc42" podUID="711f1752-03c4-4f9a-bb1b-cb7bf8cac9cd"
Sep 11 11:48:43.716768 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:43.716740    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2v285"
Sep 11 11:48:43.717824 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:43.717800    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2v285"
Sep 11 11:48:43.737271 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:43.737241    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2v285"
Sep 11 11:48:44.368974 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:44.368634    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-79ltx"]
Sep 11 11:48:44.369140 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:44.368976    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79ltx"
Sep 11 11:48:44.369140 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:44.369071    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79ltx" podUID="70994cc4-c221-4ab7-99ec-aed7c86955de"
Sep 11 11:48:44.371505 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:44.371348    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-krxsv"]
Sep 11 11:48:44.371672 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:44.371542    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-krxsv"
Sep 11 11:48:44.371752 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:44.371668    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-krxsv" podUID="fa7be6d8-e93e-49af-8782-14c8080fef33"
Sep 11 11:48:44.372150 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:44.372126    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-dqc42"]
Sep 11 11:48:44.372257 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:44.372219    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dqc42"
Sep 11 11:48:44.372316 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:44.372293    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dqc42" podUID="711f1752-03c4-4f9a-bb1b-cb7bf8cac9cd"
Sep 11 11:48:44.720450 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:44.720354    2579 generic.go:356] "Generic (PLEG): container finished" podID="7872368e-69ce-4a5c-9773-e2d02b8f06ba" containerID="e7cf392fa2f3ab52bafc1680a0c38f468d01ccd20db32be38fd0b7a11a7475d3" exitCode=0
Sep 11 11:48:44.720785 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:44.720440    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4jthr" event={"ID":"7872368e-69ce-4a5c-9773-e2d02b8f06ba","Type":"ContainerDied","Data":"e7cf392fa2f3ab52bafc1680a0c38f468d01ccd20db32be38fd0b7a11a7475d3"}
Sep 11 11:48:45.570326 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:45.570248    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79ltx"
Sep 11 11:48:45.570524 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:45.570355    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79ltx" podUID="70994cc4-c221-4ab7-99ec-aed7c86955de"
Sep 11 11:48:46.571469 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:46.571380    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dqc42"
Sep 11 11:48:46.571874 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:46.571521    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-krxsv"
Sep 11 11:48:46.571874 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:46.571517    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dqc42" podUID="711f1752-03c4-4f9a-bb1b-cb7bf8cac9cd"
Sep 11 11:48:46.571874 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:46.571626    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-krxsv" podUID="fa7be6d8-e93e-49af-8782-14c8080fef33"
Sep 11 11:48:46.728288 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:46.728257    2579 generic.go:356] "Generic (PLEG): container finished" podID="7872368e-69ce-4a5c-9773-e2d02b8f06ba" containerID="e6b6b95947ee18b6bba7b6e2ed26f8aecc805dcb66aa3d8157a036477634ff91" exitCode=0
Sep 11 11:48:46.728505 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:46.728304    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4jthr" event={"ID":"7872368e-69ce-4a5c-9773-e2d02b8f06ba","Type":"ContainerDied","Data":"e6b6b95947ee18b6bba7b6e2ed26f8aecc805dcb66aa3d8157a036477634ff91"}
Sep 11 11:48:47.569914 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:47.569882    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79ltx"
Sep 11 11:48:47.570103 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:47.569996    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79ltx" podUID="70994cc4-c221-4ab7-99ec-aed7c86955de"
Sep 11 11:48:48.569921 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:48.569884    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dqc42"
Sep 11 11:48:48.570374 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:48.569899    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-krxsv"
Sep 11 11:48:48.570374 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:48.570013    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dqc42" podUID="711f1752-03c4-4f9a-bb1b-cb7bf8cac9cd"
Sep 11 11:48:48.570374 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:48.570086    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-krxsv" podUID="fa7be6d8-e93e-49af-8782-14c8080fef33"
Sep 11 11:48:49.168595 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:49.168547    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:48:49.260483 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:49.260387    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:48:49.569834 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:49.569566    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79ltx"
Sep 11 11:48:49.569977 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:49.569909    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79ltx" podUID="70994cc4-c221-4ab7-99ec-aed7c86955de"
Sep 11 11:48:50.194666 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:50.194624    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa7be6d8-e93e-49af-8782-14c8080fef33-metrics-certs\") pod \"network-metrics-daemon-krxsv\" (UID: \"fa7be6d8-e93e-49af-8782-14c8080fef33\") " pod="openshift-multus/network-metrics-daemon-krxsv"
Sep 11 11:48:50.194831 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:50.194783    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 11:48:50.194890 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:50.194854    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa7be6d8-e93e-49af-8782-14c8080fef33-metrics-certs podName:fa7be6d8-e93e-49af-8782-14c8080fef33 nodeName:}" failed. No retries permitted until 2026-09-11 11:49:22.194835624 +0000 UTC m=+66.092277346 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fa7be6d8-e93e-49af-8782-14c8080fef33-metrics-certs") pod "network-metrics-daemon-krxsv" (UID: "fa7be6d8-e93e-49af-8782-14c8080fef33") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 11:48:50.295028 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:50.294988    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/70994cc4-c221-4ab7-99ec-aed7c86955de-original-pull-secret\") pod \"global-pull-secret-syncer-79ltx\" (UID: \"70994cc4-c221-4ab7-99ec-aed7c86955de\") " pod="kube-system/global-pull-secret-syncer-79ltx"
Sep 11 11:48:50.295223 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:50.295037    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qbg9\" (UniqueName: \"kubernetes.io/projected/711f1752-03c4-4f9a-bb1b-cb7bf8cac9cd-kube-api-access-5qbg9\") pod \"network-check-target-dqc42\" (UID: \"711f1752-03c4-4f9a-bb1b-cb7bf8cac9cd\") " pod="openshift-network-diagnostics/network-check-target-dqc42"
Sep 11 11:48:50.295223 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:50.295141    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 11:48:50.295223 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:50.295174    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 11:48:50.295223 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:50.295191    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 11:48:50.295223 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:50.295202    2579 projected.go:196] Error preparing data for projected volume kube-api-access-5qbg9 for pod openshift-network-diagnostics/network-check-target-dqc42: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 11:48:50.295223 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:50.295203    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70994cc4-c221-4ab7-99ec-aed7c86955de-original-pull-secret podName:70994cc4-c221-4ab7-99ec-aed7c86955de nodeName:}" failed. No retries permitted until 2026-09-11 11:49:22.29518549 +0000 UTC m=+66.192627226 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/70994cc4-c221-4ab7-99ec-aed7c86955de-original-pull-secret") pod "global-pull-secret-syncer-79ltx" (UID: "70994cc4-c221-4ab7-99ec-aed7c86955de") : object "kube-system"/"original-pull-secret" not registered
Sep 11 11:48:50.295523 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:50.295243    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/711f1752-03c4-4f9a-bb1b-cb7bf8cac9cd-kube-api-access-5qbg9 podName:711f1752-03c4-4f9a-bb1b-cb7bf8cac9cd nodeName:}" failed. No retries permitted until 2026-09-11 11:49:22.29523238 +0000 UTC m=+66.192674103 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-5qbg9" (UniqueName: "kubernetes.io/projected/711f1752-03c4-4f9a-bb1b-cb7bf8cac9cd-kube-api-access-5qbg9") pod "network-check-target-dqc42" (UID: "711f1752-03c4-4f9a-bb1b-cb7bf8cac9cd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 11:48:50.569825 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:50.569733    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dqc42"
Sep 11 11:48:50.570000 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:50.569733    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-krxsv"
Sep 11 11:48:50.570000 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:50.569885    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dqc42" podUID="711f1752-03c4-4f9a-bb1b-cb7bf8cac9cd"
Sep 11 11:48:50.570000 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:50.569983    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-krxsv" podUID="fa7be6d8-e93e-49af-8782-14c8080fef33"
Sep 11 11:48:51.324165 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:51.324139    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-145.ec2.internal" event="NodeReady"
Sep 11 11:48:51.324352 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:51.324250    2579 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 11 11:48:51.373761 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:51.373682    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-pmhcz"]
Sep 11 11:48:51.375556 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:51.375539    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pmhcz"
Sep 11 11:48:51.377000 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:51.376975    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-qvss2"]
Sep 11 11:48:51.378317 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:51.378297    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 11 11:48:51.378616 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:51.378596    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qvss2"
Sep 11 11:48:51.378836 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:51.378758    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 11 11:48:51.378836 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:51.378789    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-m9sw5\""
Sep 11 11:48:51.380757 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:51.380730    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 11 11:48:51.381037 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:51.381018    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 11 11:48:51.381668 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:51.381650    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 11 11:48:51.381774 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:51.381750    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-wpl6b\""
Sep 11 11:48:51.392264 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:51.392239    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qvss2"]
Sep 11 11:48:51.393081 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:51.393058    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-pmhcz"]
Sep 11 11:48:51.505496 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:51.505464    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9bc72241-e4f1-41eb-84dc-994cccf593c2-cert\") pod \"ingress-canary-qvss2\" (UID: \"9bc72241-e4f1-41eb-84dc-994cccf593c2\") " pod="openshift-ingress-canary/ingress-canary-qvss2"
Sep 11 11:48:51.505496 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:51.505495    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwbv9\" (UniqueName: \"kubernetes.io/projected/9246798b-82ff-4c57-97ff-ce4315e9a904-kube-api-access-pwbv9\") pod \"dns-default-pmhcz\" (UID: \"9246798b-82ff-4c57-97ff-ce4315e9a904\") " pod="openshift-dns/dns-default-pmhcz"
Sep 11 11:48:51.505725 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:51.505564    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9246798b-82ff-4c57-97ff-ce4315e9a904-tmp-dir\") pod \"dns-default-pmhcz\" (UID: \"9246798b-82ff-4c57-97ff-ce4315e9a904\") " pod="openshift-dns/dns-default-pmhcz"
Sep 11 11:48:51.505725 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:51.505633    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9246798b-82ff-4c57-97ff-ce4315e9a904-config-volume\") pod \"dns-default-pmhcz\" (UID: \"9246798b-82ff-4c57-97ff-ce4315e9a904\") " pod="openshift-dns/dns-default-pmhcz"
Sep 11 11:48:51.505725 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:51.505653    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9246798b-82ff-4c57-97ff-ce4315e9a904-metrics-tls\") pod \"dns-default-pmhcz\" (UID: \"9246798b-82ff-4c57-97ff-ce4315e9a904\") " pod="openshift-dns/dns-default-pmhcz"
Sep 11 11:48:51.505725 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:51.505692    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ptkr\" (UniqueName: \"kubernetes.io/projected/9bc72241-e4f1-41eb-84dc-994cccf593c2-kube-api-access-5ptkr\") pod \"ingress-canary-qvss2\" (UID: \"9bc72241-e4f1-41eb-84dc-994cccf593c2\") " pod="openshift-ingress-canary/ingress-canary-qvss2"
Sep 11 11:48:51.558635 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:51.558602    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:48:51.570052 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:51.570024    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79ltx"
Sep 11 11:48:51.572999 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:51.572983    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 11 11:48:51.606787 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:51.606752    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9bc72241-e4f1-41eb-84dc-994cccf593c2-cert\") pod \"ingress-canary-qvss2\" (UID: \"9bc72241-e4f1-41eb-84dc-994cccf593c2\") " pod="openshift-ingress-canary/ingress-canary-qvss2"
Sep 11 11:48:51.606787 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:51.606786    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwbv9\" (UniqueName: \"kubernetes.io/projected/9246798b-82ff-4c57-97ff-ce4315e9a904-kube-api-access-pwbv9\") pod \"dns-default-pmhcz\" (UID: \"9246798b-82ff-4c57-97ff-ce4315e9a904\") " pod="openshift-dns/dns-default-pmhcz"
Sep 11 11:48:51.606992 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:51.606839    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9246798b-82ff-4c57-97ff-ce4315e9a904-tmp-dir\") pod \"dns-default-pmhcz\" (UID: \"9246798b-82ff-4c57-97ff-ce4315e9a904\") " pod="openshift-dns/dns-default-pmhcz"
Sep 11 11:48:51.606992 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:51.606866    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9246798b-82ff-4c57-97ff-ce4315e9a904-config-volume\") pod \"dns-default-pmhcz\" (UID: \"9246798b-82ff-4c57-97ff-ce4315e9a904\") " pod="openshift-dns/dns-default-pmhcz"
Sep 11 11:48:51.606992 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:51.606893    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9246798b-82ff-4c57-97ff-ce4315e9a904-metrics-tls\") pod \"dns-default-pmhcz\" (UID: \"9246798b-82ff-4c57-97ff-ce4315e9a904\") " pod="openshift-dns/dns-default-pmhcz"
Sep 11 11:48:51.606992 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:51.606940    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ptkr\" (UniqueName: \"kubernetes.io/projected/9bc72241-e4f1-41eb-84dc-994cccf593c2-kube-api-access-5ptkr\") pod \"ingress-canary-qvss2\" (UID: \"9bc72241-e4f1-41eb-84dc-994cccf593c2\") " pod="openshift-ingress-canary/ingress-canary-qvss2"
Sep 11 11:48:51.607174 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:51.607091    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 11:48:51.607174 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:51.607156    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9246798b-82ff-4c57-97ff-ce4315e9a904-metrics-tls podName:9246798b-82ff-4c57-97ff-ce4315e9a904 nodeName:}" failed. No retries permitted until 2026-09-11 11:48:52.107134344 +0000 UTC m=+36.004576087 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9246798b-82ff-4c57-97ff-ce4315e9a904-metrics-tls") pod "dns-default-pmhcz" (UID: "9246798b-82ff-4c57-97ff-ce4315e9a904") : secret "dns-default-metrics-tls" not found
Sep 11 11:48:51.607174 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:51.607161    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9246798b-82ff-4c57-97ff-ce4315e9a904-tmp-dir\") pod \"dns-default-pmhcz\" (UID: \"9246798b-82ff-4c57-97ff-ce4315e9a904\") " pod="openshift-dns/dns-default-pmhcz"
Sep 11 11:48:51.607448 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:51.607430    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9246798b-82ff-4c57-97ff-ce4315e9a904-config-volume\") pod \"dns-default-pmhcz\" (UID: \"9246798b-82ff-4c57-97ff-ce4315e9a904\") " pod="openshift-dns/dns-default-pmhcz"
Sep 11 11:48:51.610924 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:51.610901    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9bc72241-e4f1-41eb-84dc-994cccf593c2-cert\") pod \"ingress-canary-qvss2\" (UID: \"9bc72241-e4f1-41eb-84dc-994cccf593c2\") " pod="openshift-ingress-canary/ingress-canary-qvss2"
Sep 11 11:48:51.616468 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:51.616446    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwbv9\" (UniqueName: \"kubernetes.io/projected/9246798b-82ff-4c57-97ff-ce4315e9a904-kube-api-access-pwbv9\") pod \"dns-default-pmhcz\" (UID: \"9246798b-82ff-4c57-97ff-ce4315e9a904\") " pod="openshift-dns/dns-default-pmhcz"
Sep 11 11:48:51.616718 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:51.616700    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ptkr\" (UniqueName: \"kubernetes.io/projected/9bc72241-e4f1-41eb-84dc-994cccf593c2-kube-api-access-5ptkr\") pod \"ingress-canary-qvss2\" (UID: \"9bc72241-e4f1-41eb-84dc-994cccf593c2\") " pod="openshift-ingress-canary/ingress-canary-qvss2"
Sep 11 11:48:51.660986 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:51.660898    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:48:51.690326 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:51.690290    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qvss2"
Sep 11 11:48:51.742472 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:51.742436    2579 generic.go:356] "Generic (PLEG): container finished" podID="7872368e-69ce-4a5c-9773-e2d02b8f06ba" containerID="8dd06b3f2416a12b04844386c437b0f93a97960cd255b0279b434780b1c71647" exitCode=0
Sep 11 11:48:51.742650 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:51.742513    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4jthr" event={"ID":"7872368e-69ce-4a5c-9773-e2d02b8f06ba","Type":"ContainerDied","Data":"8dd06b3f2416a12b04844386c437b0f93a97960cd255b0279b434780b1c71647"}
Sep 11 11:48:51.971365 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:51.971150    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qvss2"]
Sep 11 11:48:51.974585 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:48:51.974555    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bc72241_e4f1_41eb_84dc_994cccf593c2.slice/crio-1a7d36d7ec842e6ac8bfa2b0792dba8077f197ed6cee85d1d27aee8b21efedfe WatchSource:0}: Error finding container 1a7d36d7ec842e6ac8bfa2b0792dba8077f197ed6cee85d1d27aee8b21efedfe: Status 404 returned error can't find the container with id 1a7d36d7ec842e6ac8bfa2b0792dba8077f197ed6cee85d1d27aee8b21efedfe
Sep 11 11:48:52.111003 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:52.110961    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9246798b-82ff-4c57-97ff-ce4315e9a904-metrics-tls\") pod \"dns-default-pmhcz\" (UID: \"9246798b-82ff-4c57-97ff-ce4315e9a904\") " pod="openshift-dns/dns-default-pmhcz"
Sep 11 11:48:52.111169 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:52.111109    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 11:48:52.111212 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:52.111170    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9246798b-82ff-4c57-97ff-ce4315e9a904-metrics-tls podName:9246798b-82ff-4c57-97ff-ce4315e9a904 nodeName:}" failed. No retries permitted until 2026-09-11 11:48:53.11115593 +0000 UTC m=+37.008597672 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9246798b-82ff-4c57-97ff-ce4315e9a904-metrics-tls") pod "dns-default-pmhcz" (UID: "9246798b-82ff-4c57-97ff-ce4315e9a904") : secret "dns-default-metrics-tls" not found
Sep 11 11:48:52.570186 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:52.570156    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dqc42"
Sep 11 11:48:52.570980 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:52.570364    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-krxsv"
Sep 11 11:48:52.575115 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:52.575095    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5sht6\""
Sep 11 11:48:52.575242 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:52.575190    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 11 11:48:52.575242 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:52.575197    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 11 11:48:52.575358 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:52.575293    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 11 11:48:52.575437 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:52.575361    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-xblbl\""
Sep 11 11:48:52.744701 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:52.744666    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:48:52.746240 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:52.746209    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qvss2" event={"ID":"9bc72241-e4f1-41eb-84dc-994cccf593c2","Type":"ContainerStarted","Data":"1a7d36d7ec842e6ac8bfa2b0792dba8077f197ed6cee85d1d27aee8b21efedfe"}
Sep 11 11:48:52.749135 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:52.749112    2579 generic.go:356] "Generic (PLEG): container finished" podID="7872368e-69ce-4a5c-9773-e2d02b8f06ba" containerID="bd12f97577e1f189640f10f1a8df5870db7ae285b1cd8d5355e75d9b95be5f12" exitCode=0
Sep 11 11:48:52.749241 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:52.749146    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4jthr" event={"ID":"7872368e-69ce-4a5c-9773-e2d02b8f06ba","Type":"ContainerDied","Data":"bd12f97577e1f189640f10f1a8df5870db7ae285b1cd8d5355e75d9b95be5f12"}
Sep 11 11:48:53.116586 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:53.116553    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9246798b-82ff-4c57-97ff-ce4315e9a904-metrics-tls\") pod \"dns-default-pmhcz\" (UID: \"9246798b-82ff-4c57-97ff-ce4315e9a904\") " pod="openshift-dns/dns-default-pmhcz"
Sep 11 11:48:53.116763 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:53.116708    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 11:48:53.116802 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:53.116771    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9246798b-82ff-4c57-97ff-ce4315e9a904-metrics-tls podName:9246798b-82ff-4c57-97ff-ce4315e9a904 nodeName:}" failed. No retries permitted until 2026-09-11 11:48:55.116756743 +0000 UTC m=+39.014198465 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9246798b-82ff-4c57-97ff-ce4315e9a904-metrics-tls") pod "dns-default-pmhcz" (UID: "9246798b-82ff-4c57-97ff-ce4315e9a904") : secret "dns-default-metrics-tls" not found
Sep 11 11:48:53.755789 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:53.755757    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4jthr" event={"ID":"7872368e-69ce-4a5c-9773-e2d02b8f06ba","Type":"ContainerStarted","Data":"b11e016ae6bdb934c6b5312395358a17b23b8f9597d1c7af4c39ebb34fe91709"}
Sep 11 11:48:53.781345 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:53.781256    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-4jthr" podStartSLOduration=6.81668901 podStartE2EDuration="36.781235386s" totalImagesPullingTime="29.964546376s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 11:48:17 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:48:53.779660673 +0000 UTC m=+37.677102428" watchObservedRunningTime="2026-09-11 11:48:53.781235386 +0000 UTC m=+37.678677132"
Sep 11 11:48:53.968807 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:53.968760    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:48:54.038012 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:54.037907    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:48:54.427870 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:54.427834    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:48:54.527726 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:54.527684    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:48:55.133618 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:55.133577    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9246798b-82ff-4c57-97ff-ce4315e9a904-metrics-tls\") pod \"dns-default-pmhcz\" (UID: \"9246798b-82ff-4c57-97ff-ce4315e9a904\") " pod="openshift-dns/dns-default-pmhcz"
Sep 11 11:48:55.134073 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:55.133759    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 11:48:55.134073 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:55.133840    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9246798b-82ff-4c57-97ff-ce4315e9a904-metrics-tls podName:9246798b-82ff-4c57-97ff-ce4315e9a904 nodeName:}" failed. No retries permitted until 2026-09-11 11:48:59.133817816 +0000 UTC m=+43.031259562 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9246798b-82ff-4c57-97ff-ce4315e9a904-metrics-tls") pod "dns-default-pmhcz" (UID: "9246798b-82ff-4c57-97ff-ce4315e9a904") : secret "dns-default-metrics-tls" not found
Sep 11 11:48:56.764482 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:56.764442    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qvss2" event={"ID":"9bc72241-e4f1-41eb-84dc-994cccf593c2","Type":"ContainerStarted","Data":"791ce34c468d7cd3599313df68fb218becf0903cf0d501f7d61106f475b75053"}
Sep 11 11:48:56.784856 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:56.784808    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-qvss2" podStartSLOduration=2.063499995 podStartE2EDuration="5.78479298s" totalImagesPullingTime="3.721292985s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:48:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:48:56.78425741 +0000 UTC m=+40.681699155" watchObservedRunningTime="2026-09-11 11:48:56.78479298 +0000 UTC m=+40.682234745"
Sep 11 11:48:57.445448 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:57.445409    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:48:57.599318 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:57.599281    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:48:59.159958 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:48:59.159918    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9246798b-82ff-4c57-97ff-ce4315e9a904-metrics-tls\") pod \"dns-default-pmhcz\" (UID: \"9246798b-82ff-4c57-97ff-ce4315e9a904\") " pod="openshift-dns/dns-default-pmhcz"
Sep 11 11:48:59.160469 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:59.160088    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 11:48:59.160469 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:48:59.160165    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9246798b-82ff-4c57-97ff-ce4315e9a904-metrics-tls podName:9246798b-82ff-4c57-97ff-ce4315e9a904 nodeName:}" failed. No retries permitted until 2026-09-11 11:49:07.160146499 +0000 UTC m=+51.057588223 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9246798b-82ff-4c57-97ff-ce4315e9a904-metrics-tls") pod "dns-default-pmhcz" (UID: "9246798b-82ff-4c57-97ff-ce4315e9a904") : secret "dns-default-metrics-tls" not found
Sep 11 11:49:07.208776 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:49:07.208736    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9246798b-82ff-4c57-97ff-ce4315e9a904-metrics-tls\") pod \"dns-default-pmhcz\" (UID: \"9246798b-82ff-4c57-97ff-ce4315e9a904\") " pod="openshift-dns/dns-default-pmhcz"
Sep 11 11:49:07.209221 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:49:07.208894    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 11:49:07.209221 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:49:07.208973    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9246798b-82ff-4c57-97ff-ce4315e9a904-metrics-tls podName:9246798b-82ff-4c57-97ff-ce4315e9a904 nodeName:}" failed. No retries permitted until 2026-09-11 11:49:23.208957979 +0000 UTC m=+67.106399706 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9246798b-82ff-4c57-97ff-ce4315e9a904-metrics-tls") pod "dns-default-pmhcz" (UID: "9246798b-82ff-4c57-97ff-ce4315e9a904") : secret "dns-default-metrics-tls" not found
Sep 11 11:49:15.738083 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:49:15.738050    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2v285"
Sep 11 11:49:15.888519 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:49:15.888481    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:49:15.990722 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:49:15.990634    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:49:22.211084 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:49:22.211039    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa7be6d8-e93e-49af-8782-14c8080fef33-metrics-certs\") pod \"network-metrics-daemon-krxsv\" (UID: \"fa7be6d8-e93e-49af-8782-14c8080fef33\") " pod="openshift-multus/network-metrics-daemon-krxsv"
Sep 11 11:49:22.215052 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:49:22.215026    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 11 11:49:22.221696 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:49:22.221675    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 11 11:49:22.221801 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:49:22.221747    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa7be6d8-e93e-49af-8782-14c8080fef33-metrics-certs podName:fa7be6d8-e93e-49af-8782-14c8080fef33 nodeName:}" failed. No retries permitted until 2026-09-11 11:50:26.221732165 +0000 UTC m=+130.119173894 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fa7be6d8-e93e-49af-8782-14c8080fef33-metrics-certs") pod "network-metrics-daemon-krxsv" (UID: "fa7be6d8-e93e-49af-8782-14c8080fef33") : secret "metrics-daemon-secret" not found
Sep 11 11:49:22.311671 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:49:22.311636    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/70994cc4-c221-4ab7-99ec-aed7c86955de-original-pull-secret\") pod \"global-pull-secret-syncer-79ltx\" (UID: \"70994cc4-c221-4ab7-99ec-aed7c86955de\") " pod="kube-system/global-pull-secret-syncer-79ltx"
Sep 11 11:49:22.311671 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:49:22.311677    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qbg9\" (UniqueName: \"kubernetes.io/projected/711f1752-03c4-4f9a-bb1b-cb7bf8cac9cd-kube-api-access-5qbg9\") pod \"network-check-target-dqc42\" (UID: \"711f1752-03c4-4f9a-bb1b-cb7bf8cac9cd\") " pod="openshift-network-diagnostics/network-check-target-dqc42"
Sep 11 11:49:22.314716 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:49:22.314697    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 11 11:49:22.314775 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:49:22.314732    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 11 11:49:22.324729 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:49:22.324698    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 11 11:49:22.325477 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:49:22.325453    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/70994cc4-c221-4ab7-99ec-aed7c86955de-original-pull-secret\") pod \"global-pull-secret-syncer-79ltx\" (UID: \"70994cc4-c221-4ab7-99ec-aed7c86955de\") " pod="kube-system/global-pull-secret-syncer-79ltx"
Sep 11 11:49:22.334990 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:49:22.334960    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qbg9\" (UniqueName: \"kubernetes.io/projected/711f1752-03c4-4f9a-bb1b-cb7bf8cac9cd-kube-api-access-5qbg9\") pod \"network-check-target-dqc42\" (UID: \"711f1752-03c4-4f9a-bb1b-cb7bf8cac9cd\") " pod="openshift-network-diagnostics/network-check-target-dqc42"
Sep 11 11:49:22.478685 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:49:22.478603    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79ltx"
Sep 11 11:49:22.583693 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:49:22.583664    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-xblbl\""
Sep 11 11:49:22.591128 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:49:22.591107    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dqc42"
Sep 11 11:49:22.780537 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:49:22.780461    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-79ltx"]
Sep 11 11:49:22.783527 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:49:22.783503    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70994cc4_c221_4ab7_99ec_aed7c86955de.slice/crio-65a98c6f7657df4cd12ac3a5f69d90fab61a6e940b3be162d5a6a1c37ef2f80d WatchSource:0}: Error finding container 65a98c6f7657df4cd12ac3a5f69d90fab61a6e940b3be162d5a6a1c37ef2f80d: Status 404 returned error can't find the container with id 65a98c6f7657df4cd12ac3a5f69d90fab61a6e940b3be162d5a6a1c37ef2f80d
Sep 11 11:49:22.828428 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:49:22.828378    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-79ltx" event={"ID":"70994cc4-c221-4ab7-99ec-aed7c86955de","Type":"ContainerStarted","Data":"65a98c6f7657df4cd12ac3a5f69d90fab61a6e940b3be162d5a6a1c37ef2f80d"}
Sep 11 11:49:22.881507 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:49:22.881477    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-dqc42"]
Sep 11 11:49:22.884625 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:49:22.884604    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod711f1752_03c4_4f9a_bb1b_cb7bf8cac9cd.slice/crio-d7ea02dd349501f1ce9c191b31af70463ca4eec7d0862e8ea88d9d611c362ef5 WatchSource:0}: Error finding container d7ea02dd349501f1ce9c191b31af70463ca4eec7d0862e8ea88d9d611c362ef5: Status 404 returned error can't find the container with id d7ea02dd349501f1ce9c191b31af70463ca4eec7d0862e8ea88d9d611c362ef5
Sep 11 11:49:23.217566 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:49:23.217520    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9246798b-82ff-4c57-97ff-ce4315e9a904-metrics-tls\") pod \"dns-default-pmhcz\" (UID: \"9246798b-82ff-4c57-97ff-ce4315e9a904\") " pod="openshift-dns/dns-default-pmhcz"
Sep 11 11:49:23.217961 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:49:23.217679    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 11:49:23.217961 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:49:23.217745    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9246798b-82ff-4c57-97ff-ce4315e9a904-metrics-tls podName:9246798b-82ff-4c57-97ff-ce4315e9a904 nodeName:}" failed. No retries permitted until 2026-09-11 11:49:55.217728234 +0000 UTC m=+99.115169957 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9246798b-82ff-4c57-97ff-ce4315e9a904-metrics-tls") pod "dns-default-pmhcz" (UID: "9246798b-82ff-4c57-97ff-ce4315e9a904") : secret "dns-default-metrics-tls" not found
Sep 11 11:49:23.832590 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:49:23.832548    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-dqc42" event={"ID":"711f1752-03c4-4f9a-bb1b-cb7bf8cac9cd","Type":"ContainerStarted","Data":"d7ea02dd349501f1ce9c191b31af70463ca4eec7d0862e8ea88d9d611c362ef5"}
Sep 11 11:49:27.844351 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:49:27.844314    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-79ltx" event={"ID":"70994cc4-c221-4ab7-99ec-aed7c86955de","Type":"ContainerStarted","Data":"0f8200c89ddea845adaba72ce6c99c90c33a6bbae8920c73ce4b84fb48df8300"}
Sep 11 11:49:27.845881 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:49:27.845854    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-dqc42" event={"ID":"711f1752-03c4-4f9a-bb1b-cb7bf8cac9cd","Type":"ContainerStarted","Data":"b0c39c93232a12c53379e9bd0d349bc7991687b441fb29e989bbf53a9ee6063f"}
Sep 11 11:49:27.846034 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:49:27.845985    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-dqc42"
Sep 11 11:49:27.862846 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:49:27.862790    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-79ltx" podStartSLOduration=66.166818876 podStartE2EDuration="1m10.862770366s" totalImagesPullingTime="4.69595149s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 11:48:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:49:27.862733315 +0000 UTC m=+71.760175060" watchObservedRunningTime="2026-09-11 11:49:27.862770366 +0000 UTC m=+71.760212089"
Sep 11 11:49:27.879663 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:49:27.879611    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-dqc42" podStartSLOduration=67.288567083 podStartE2EDuration="1m11.879598231s" totalImagesPullingTime="4.591031148s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:48:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:49:27.879272536 +0000 UTC m=+71.776714281" watchObservedRunningTime="2026-09-11 11:49:27.879598231 +0000 UTC m=+71.777039973"
Sep 11 11:49:55.318216 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:49:55.318173    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9246798b-82ff-4c57-97ff-ce4315e9a904-metrics-tls\") pod \"dns-default-pmhcz\" (UID: \"9246798b-82ff-4c57-97ff-ce4315e9a904\") " pod="openshift-dns/dns-default-pmhcz"
Sep 11 11:49:55.318685 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:49:55.318289    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 11:49:55.318685 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:49:55.318339    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9246798b-82ff-4c57-97ff-ce4315e9a904-metrics-tls podName:9246798b-82ff-4c57-97ff-ce4315e9a904 nodeName:}" failed. No retries permitted until 2026-09-11 11:50:59.318326918 +0000 UTC m=+163.215768641 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9246798b-82ff-4c57-97ff-ce4315e9a904-metrics-tls") pod "dns-default-pmhcz" (UID: "9246798b-82ff-4c57-97ff-ce4315e9a904") : secret "dns-default-metrics-tls" not found
Sep 11 11:49:58.850734 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:49:58.850701    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-dqc42"
Sep 11 11:50:04.510659 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.510614    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-ch96n"]
Sep 11 11:50:04.513256 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.513239    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-ch96n"
Sep 11 11:50:04.516608 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.516584    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 11 11:50:04.517017 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.516995    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 11 11:50:04.517125 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.517021    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-r5q8c\""
Sep 11 11:50:04.517814 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.517798    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 11 11:50:04.519659 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.519297    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 11 11:50:04.525009 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.524985    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-ch96n"]
Sep 11 11:50:04.525942 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.525919    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 11 11:50:04.583060 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.583025    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1f0f9fa5-cbd1-468b-ae81-82b48c101d37-serving-cert\") pod \"console-operator-696884fb9c-ch96n\" (UID: \"1f0f9fa5-cbd1-468b-ae81-82b48c101d37\") " pod="openshift-console-operator/console-operator-696884fb9c-ch96n"
Sep 11 11:50:04.583220 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.583133    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wcfsm\" (UniqueName: \"kubernetes.io/projected/1f0f9fa5-cbd1-468b-ae81-82b48c101d37-kube-api-access-wcfsm\") pod \"console-operator-696884fb9c-ch96n\" (UID: \"1f0f9fa5-cbd1-468b-ae81-82b48c101d37\") " pod="openshift-console-operator/console-operator-696884fb9c-ch96n"
Sep 11 11:50:04.583220 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.583165    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1f0f9fa5-cbd1-468b-ae81-82b48c101d37-config\") pod \"console-operator-696884fb9c-ch96n\" (UID: \"1f0f9fa5-cbd1-468b-ae81-82b48c101d37\") " pod="openshift-console-operator/console-operator-696884fb9c-ch96n"
Sep 11 11:50:04.583220 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.583191    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1f0f9fa5-cbd1-468b-ae81-82b48c101d37-trusted-ca\") pod \"console-operator-696884fb9c-ch96n\" (UID: \"1f0f9fa5-cbd1-468b-ae81-82b48c101d37\") " pod="openshift-console-operator/console-operator-696884fb9c-ch96n"
Sep 11 11:50:04.629376 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.629340    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-7b5c4b4df6-24w55"]
Sep 11 11:50:04.631198 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.631183    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5844b856db-9l6kz"]
Sep 11 11:50:04.631325 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.631305    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7b5c4b4df6-24w55"
Sep 11 11:50:04.633786 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.633762    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-9l6kz"
Sep 11 11:50:04.636376 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.636352    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 11 11:50:04.636508 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.636407    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 11 11:50:04.636687 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.636672    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 11 11:50:04.636928 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.636912    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-5ld9w\""
Sep 11 11:50:04.637454 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.637436    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 11 11:50:04.637541 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.637459    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 11 11:50:04.637996 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.637972    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 11 11:50:04.638300 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.638281    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 11 11:50:04.638539 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.638521    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 11 11:50:04.638643 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.638604    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 11 11:50:04.638878 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.638862    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-4wgbq\""
Sep 11 11:50:04.638952 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.638917    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 11 11:50:04.643176 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.643157    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 11 11:50:04.647777 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.647746    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-9l6kz"]
Sep 11 11:50:04.648538 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.648510    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7b5c4b4df6-24w55"]
Sep 11 11:50:04.684415 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.684361    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7f0c67d9-ee3f-4f68-9e53-2b6e8b238d5e-tmp\") pod \"insights-operator-5844b856db-9l6kz\" (UID: \"7f0c67d9-ee3f-4f68-9e53-2b6e8b238d5e\") " pod="openshift-insights/insights-operator-5844b856db-9l6kz"
Sep 11 11:50:04.684595 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.684421    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkgz5\" (UniqueName: \"kubernetes.io/projected/7f0c67d9-ee3f-4f68-9e53-2b6e8b238d5e-kube-api-access-rkgz5\") pod \"insights-operator-5844b856db-9l6kz\" (UID: \"7f0c67d9-ee3f-4f68-9e53-2b6e8b238d5e\") " pod="openshift-insights/insights-operator-5844b856db-9l6kz"
Sep 11 11:50:04.684595 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.684499    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e8b4926-8dc4-4622-9882-8d039646c1c0-service-ca-bundle\") pod \"router-default-7b5c4b4df6-24w55\" (UID: \"6e8b4926-8dc4-4622-9882-8d039646c1c0\") " pod="openshift-ingress/router-default-7b5c4b4df6-24w55"
Sep 11 11:50:04.684595 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.684525    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/7f0c67d9-ee3f-4f68-9e53-2b6e8b238d5e-snapshots\") pod \"insights-operator-5844b856db-9l6kz\" (UID: \"7f0c67d9-ee3f-4f68-9e53-2b6e8b238d5e\") " pod="openshift-insights/insights-operator-5844b856db-9l6kz"
Sep 11 11:50:04.684595 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.684561    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/6e8b4926-8dc4-4622-9882-8d039646c1c0-default-certificate\") pod \"router-default-7b5c4b4df6-24w55\" (UID: \"6e8b4926-8dc4-4622-9882-8d039646c1c0\") " pod="openshift-ingress/router-default-7b5c4b4df6-24w55"
Sep 11 11:50:04.684595 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.684580    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4l4hq\" (UniqueName: \"kubernetes.io/projected/6e8b4926-8dc4-4622-9882-8d039646c1c0-kube-api-access-4l4hq\") pod \"router-default-7b5c4b4df6-24w55\" (UID: \"6e8b4926-8dc4-4622-9882-8d039646c1c0\") " pod="openshift-ingress/router-default-7b5c4b4df6-24w55"
Sep 11 11:50:04.684595 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.684595    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f0c67d9-ee3f-4f68-9e53-2b6e8b238d5e-service-ca-bundle\") pod \"insights-operator-5844b856db-9l6kz\" (UID: \"7f0c67d9-ee3f-4f68-9e53-2b6e8b238d5e\") " pod="openshift-insights/insights-operator-5844b856db-9l6kz"
Sep 11 11:50:04.684954 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.684639    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcfsm\" (UniqueName: \"kubernetes.io/projected/1f0f9fa5-cbd1-468b-ae81-82b48c101d37-kube-api-access-wcfsm\") pod \"console-operator-696884fb9c-ch96n\" (UID: \"1f0f9fa5-cbd1-468b-ae81-82b48c101d37\") " pod="openshift-console-operator/console-operator-696884fb9c-ch96n"
Sep 11 11:50:04.684954 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.684689    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f0c67d9-ee3f-4f68-9e53-2b6e8b238d5e-trusted-ca-bundle\") pod \"insights-operator-5844b856db-9l6kz\" (UID: \"7f0c67d9-ee3f-4f68-9e53-2b6e8b238d5e\") " pod="openshift-insights/insights-operator-5844b856db-9l6kz"
Sep 11 11:50:04.684954 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.684726    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1f0f9fa5-cbd1-468b-ae81-82b48c101d37-config\") pod \"console-operator-696884fb9c-ch96n\" (UID: \"1f0f9fa5-cbd1-468b-ae81-82b48c101d37\") " pod="openshift-console-operator/console-operator-696884fb9c-ch96n"
Sep 11 11:50:04.684954 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.684753    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1f0f9fa5-cbd1-468b-ae81-82b48c101d37-trusted-ca\") pod \"console-operator-696884fb9c-ch96n\" (UID: \"1f0f9fa5-cbd1-468b-ae81-82b48c101d37\") " pod="openshift-console-operator/console-operator-696884fb9c-ch96n"
Sep 11 11:50:04.684954 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.684782    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1f0f9fa5-cbd1-468b-ae81-82b48c101d37-serving-cert\") pod \"console-operator-696884fb9c-ch96n\" (UID: \"1f0f9fa5-cbd1-468b-ae81-82b48c101d37\") " pod="openshift-console-operator/console-operator-696884fb9c-ch96n"
Sep 11 11:50:04.684954 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.684810    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/6e8b4926-8dc4-4622-9882-8d039646c1c0-stats-auth\") pod \"router-default-7b5c4b4df6-24w55\" (UID: \"6e8b4926-8dc4-4622-9882-8d039646c1c0\") " pod="openshift-ingress/router-default-7b5c4b4df6-24w55"
Sep 11 11:50:04.684954 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.684834    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6e8b4926-8dc4-4622-9882-8d039646c1c0-metrics-certs\") pod \"router-default-7b5c4b4df6-24w55\" (UID: \"6e8b4926-8dc4-4622-9882-8d039646c1c0\") " pod="openshift-ingress/router-default-7b5c4b4df6-24w55"
Sep 11 11:50:04.684954 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.684904    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7f0c67d9-ee3f-4f68-9e53-2b6e8b238d5e-serving-cert\") pod \"insights-operator-5844b856db-9l6kz\" (UID: \"7f0c67d9-ee3f-4f68-9e53-2b6e8b238d5e\") " pod="openshift-insights/insights-operator-5844b856db-9l6kz"
Sep 11 11:50:04.685469 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.685452    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1f0f9fa5-cbd1-468b-ae81-82b48c101d37-config\") pod \"console-operator-696884fb9c-ch96n\" (UID: \"1f0f9fa5-cbd1-468b-ae81-82b48c101d37\") " pod="openshift-console-operator/console-operator-696884fb9c-ch96n"
Sep 11 11:50:04.685638 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.685621    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1f0f9fa5-cbd1-468b-ae81-82b48c101d37-trusted-ca\") pod \"console-operator-696884fb9c-ch96n\" (UID: \"1f0f9fa5-cbd1-468b-ae81-82b48c101d37\") " pod="openshift-console-operator/console-operator-696884fb9c-ch96n"
Sep 11 11:50:04.687014 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.686997    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1f0f9fa5-cbd1-468b-ae81-82b48c101d37-serving-cert\") pod \"console-operator-696884fb9c-ch96n\" (UID: \"1f0f9fa5-cbd1-468b-ae81-82b48c101d37\") " pod="openshift-console-operator/console-operator-696884fb9c-ch96n"
Sep 11 11:50:04.696703 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.696670    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wcfsm\" (UniqueName: \"kubernetes.io/projected/1f0f9fa5-cbd1-468b-ae81-82b48c101d37-kube-api-access-wcfsm\") pod \"console-operator-696884fb9c-ch96n\" (UID: \"1f0f9fa5-cbd1-468b-ae81-82b48c101d37\") " pod="openshift-console-operator/console-operator-696884fb9c-ch96n"
Sep 11 11:50:04.711143 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.711112    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-t4gdk"]
Sep 11 11:50:04.713020 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.713004    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-t4gdk"
Sep 11 11:50:04.716439 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.716417    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 11 11:50:04.716906 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.716886    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 11 11:50:04.717714 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.717694    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 11 11:50:04.717806 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.717713    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-g7q69\""
Sep 11 11:50:04.717806 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.717699    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 11 11:50:04.724979 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.724958    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-twxgv"]
Sep 11 11:50:04.727026 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.727007    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-t4gdk"]
Sep 11 11:50:04.727109 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.727086    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-twxgv"
Sep 11 11:50:04.729878 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.729855    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-jx6cw\""
Sep 11 11:50:04.739118 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.739090    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-twxgv"]
Sep 11 11:50:04.785323 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.785228    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e8b4926-8dc4-4622-9882-8d039646c1c0-service-ca-bundle\") pod \"router-default-7b5c4b4df6-24w55\" (UID: \"6e8b4926-8dc4-4622-9882-8d039646c1c0\") " pod="openshift-ingress/router-default-7b5c4b4df6-24w55"
Sep 11 11:50:04.785323 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.785269    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/7f0c67d9-ee3f-4f68-9e53-2b6e8b238d5e-snapshots\") pod \"insights-operator-5844b856db-9l6kz\" (UID: \"7f0c67d9-ee3f-4f68-9e53-2b6e8b238d5e\") " pod="openshift-insights/insights-operator-5844b856db-9l6kz"
Sep 11 11:50:04.785323 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.785298    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a3d1c5cd-c0aa-446c-91a2-1df64839bf16-serving-cert\") pod \"service-ca-operator-cf7c776b7-t4gdk\" (UID: \"a3d1c5cd-c0aa-446c-91a2-1df64839bf16\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-t4gdk"
Sep 11 11:50:04.785617 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.785343    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/6e8b4926-8dc4-4622-9882-8d039646c1c0-default-certificate\") pod \"router-default-7b5c4b4df6-24w55\" (UID: \"6e8b4926-8dc4-4622-9882-8d039646c1c0\") " pod="openshift-ingress/router-default-7b5c4b4df6-24w55"
Sep 11 11:50:04.785617 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.785368    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4l4hq\" (UniqueName: \"kubernetes.io/projected/6e8b4926-8dc4-4622-9882-8d039646c1c0-kube-api-access-4l4hq\") pod \"router-default-7b5c4b4df6-24w55\" (UID: \"6e8b4926-8dc4-4622-9882-8d039646c1c0\") " pod="openshift-ingress/router-default-7b5c4b4df6-24w55"
Sep 11 11:50:04.785617 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:50:04.785425    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6e8b4926-8dc4-4622-9882-8d039646c1c0-service-ca-bundle podName:6e8b4926-8dc4-4622-9882-8d039646c1c0 nodeName:}" failed. No retries permitted until 2026-09-11 11:50:05.285384026 +0000 UTC m=+109.182825768 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6e8b4926-8dc4-4622-9882-8d039646c1c0-service-ca-bundle") pod "router-default-7b5c4b4df6-24w55" (UID: "6e8b4926-8dc4-4622-9882-8d039646c1c0") : configmap references non-existent config key: service-ca.crt
Sep 11 11:50:04.785617 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.785450    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f0c67d9-ee3f-4f68-9e53-2b6e8b238d5e-service-ca-bundle\") pod \"insights-operator-5844b856db-9l6kz\" (UID: \"7f0c67d9-ee3f-4f68-9e53-2b6e8b238d5e\") " pod="openshift-insights/insights-operator-5844b856db-9l6kz"
Sep 11 11:50:04.785617 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.785487    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f0c67d9-ee3f-4f68-9e53-2b6e8b238d5e-trusted-ca-bundle\") pod \"insights-operator-5844b856db-9l6kz\" (UID: \"7f0c67d9-ee3f-4f68-9e53-2b6e8b238d5e\") " pod="openshift-insights/insights-operator-5844b856db-9l6kz"
Sep 11 11:50:04.785617 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.785525    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8c6sp\" (UniqueName: \"kubernetes.io/projected/26a684aa-1e23-44b8-a876-7ee9bbf2709c-kube-api-access-8c6sp\") pod \"network-check-source-6f8b768455-twxgv\" (UID: \"26a684aa-1e23-44b8-a876-7ee9bbf2709c\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-twxgv"
Sep 11 11:50:04.785617 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.785554    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a3d1c5cd-c0aa-446c-91a2-1df64839bf16-config\") pod \"service-ca-operator-cf7c776b7-t4gdk\" (UID: \"a3d1c5cd-c0aa-446c-91a2-1df64839bf16\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-t4gdk"
Sep 11 11:50:04.785989 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.785666    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/6e8b4926-8dc4-4622-9882-8d039646c1c0-stats-auth\") pod \"router-default-7b5c4b4df6-24w55\" (UID: \"6e8b4926-8dc4-4622-9882-8d039646c1c0\") " pod="openshift-ingress/router-default-7b5c4b4df6-24w55"
Sep 11 11:50:04.786040 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.785984    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6e8b4926-8dc4-4622-9882-8d039646c1c0-metrics-certs\") pod \"router-default-7b5c4b4df6-24w55\" (UID: \"6e8b4926-8dc4-4622-9882-8d039646c1c0\") " pod="openshift-ingress/router-default-7b5c4b4df6-24w55"
Sep 11 11:50:04.786093 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.786075    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7f0c67d9-ee3f-4f68-9e53-2b6e8b238d5e-serving-cert\") pod \"insights-operator-5844b856db-9l6kz\" (UID: \"7f0c67d9-ee3f-4f68-9e53-2b6e8b238d5e\") " pod="openshift-insights/insights-operator-5844b856db-9l6kz"
Sep 11 11:50:04.786339 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.786310    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f0c67d9-ee3f-4f68-9e53-2b6e8b238d5e-service-ca-bundle\") pod \"insights-operator-5844b856db-9l6kz\" (UID: \"7f0c67d9-ee3f-4f68-9e53-2b6e8b238d5e\") " pod="openshift-insights/insights-operator-5844b856db-9l6kz"
Sep 11 11:50:04.786453 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.786341    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/7f0c67d9-ee3f-4f68-9e53-2b6e8b238d5e-snapshots\") pod \"insights-operator-5844b856db-9l6kz\" (UID: \"7f0c67d9-ee3f-4f68-9e53-2b6e8b238d5e\") " pod="openshift-insights/insights-operator-5844b856db-9l6kz"
Sep 11 11:50:04.786615 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:50:04.786595    2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 11 11:50:04.786693 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:50:04.786678    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e8b4926-8dc4-4622-9882-8d039646c1c0-metrics-certs podName:6e8b4926-8dc4-4622-9882-8d039646c1c0 nodeName:}" failed. No retries permitted until 2026-09-11 11:50:05.286651731 +0000 UTC m=+109.184093454 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6e8b4926-8dc4-4622-9882-8d039646c1c0-metrics-certs") pod "router-default-7b5c4b4df6-24w55" (UID: "6e8b4926-8dc4-4622-9882-8d039646c1c0") : secret "router-metrics-certs-default" not found
Sep 11 11:50:04.786756 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.786116    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrltp\" (UniqueName: \"kubernetes.io/projected/a3d1c5cd-c0aa-446c-91a2-1df64839bf16-kube-api-access-xrltp\") pod \"service-ca-operator-cf7c776b7-t4gdk\" (UID: \"a3d1c5cd-c0aa-446c-91a2-1df64839bf16\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-t4gdk"
Sep 11 11:50:04.786816 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.786788    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f0c67d9-ee3f-4f68-9e53-2b6e8b238d5e-trusted-ca-bundle\") pod \"insights-operator-5844b856db-9l6kz\" (UID: \"7f0c67d9-ee3f-4f68-9e53-2b6e8b238d5e\") " pod="openshift-insights/insights-operator-5844b856db-9l6kz"
Sep 11 11:50:04.786816 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.786792    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7f0c67d9-ee3f-4f68-9e53-2b6e8b238d5e-tmp\") pod \"insights-operator-5844b856db-9l6kz\" (UID: \"7f0c67d9-ee3f-4f68-9e53-2b6e8b238d5e\") " pod="openshift-insights/insights-operator-5844b856db-9l6kz"
Sep 11 11:50:04.786922 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.786834    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkgz5\" (UniqueName: \"kubernetes.io/projected/7f0c67d9-ee3f-4f68-9e53-2b6e8b238d5e-kube-api-access-rkgz5\") pod \"insights-operator-5844b856db-9l6kz\" (UID: \"7f0c67d9-ee3f-4f68-9e53-2b6e8b238d5e\") " pod="openshift-insights/insights-operator-5844b856db-9l6kz"
Sep 11 11:50:04.788850 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.788827    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7f0c67d9-ee3f-4f68-9e53-2b6e8b238d5e-tmp\") pod \"insights-operator-5844b856db-9l6kz\" (UID: \"7f0c67d9-ee3f-4f68-9e53-2b6e8b238d5e\") " pod="openshift-insights/insights-operator-5844b856db-9l6kz"
Sep 11 11:50:04.788967 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.788930    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7f0c67d9-ee3f-4f68-9e53-2b6e8b238d5e-serving-cert\") pod \"insights-operator-5844b856db-9l6kz\" (UID: \"7f0c67d9-ee3f-4f68-9e53-2b6e8b238d5e\") " pod="openshift-insights/insights-operator-5844b856db-9l6kz"
Sep 11 11:50:04.789076 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.789053    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/6e8b4926-8dc4-4622-9882-8d039646c1c0-default-certificate\") pod \"router-default-7b5c4b4df6-24w55\" (UID: \"6e8b4926-8dc4-4622-9882-8d039646c1c0\") " pod="openshift-ingress/router-default-7b5c4b4df6-24w55"
Sep 11 11:50:04.791760 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.791738    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/6e8b4926-8dc4-4622-9882-8d039646c1c0-stats-auth\") pod \"router-default-7b5c4b4df6-24w55\" (UID: \"6e8b4926-8dc4-4622-9882-8d039646c1c0\") " pod="openshift-ingress/router-default-7b5c4b4df6-24w55"
Sep 11 11:50:04.795649 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.795628    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4l4hq\" (UniqueName: \"kubernetes.io/projected/6e8b4926-8dc4-4622-9882-8d039646c1c0-kube-api-access-4l4hq\") pod \"router-default-7b5c4b4df6-24w55\" (UID: \"6e8b4926-8dc4-4622-9882-8d039646c1c0\") " pod="openshift-ingress/router-default-7b5c4b4df6-24w55"
Sep 11 11:50:04.798066 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.798040    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkgz5\" (UniqueName: \"kubernetes.io/projected/7f0c67d9-ee3f-4f68-9e53-2b6e8b238d5e-kube-api-access-rkgz5\") pod \"insights-operator-5844b856db-9l6kz\" (UID: \"7f0c67d9-ee3f-4f68-9e53-2b6e8b238d5e\") " pod="openshift-insights/insights-operator-5844b856db-9l6kz"
Sep 11 11:50:04.823702 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.823664    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-ch96n"
Sep 11 11:50:04.832632 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.832602    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:04.887351 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.887314    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a3d1c5cd-c0aa-446c-91a2-1df64839bf16-serving-cert\") pod \"service-ca-operator-cf7c776b7-t4gdk\" (UID: \"a3d1c5cd-c0aa-446c-91a2-1df64839bf16\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-t4gdk"
Sep 11 11:50:04.887803 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.887425    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8c6sp\" (UniqueName: \"kubernetes.io/projected/26a684aa-1e23-44b8-a876-7ee9bbf2709c-kube-api-access-8c6sp\") pod \"network-check-source-6f8b768455-twxgv\" (UID: \"26a684aa-1e23-44b8-a876-7ee9bbf2709c\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-twxgv"
Sep 11 11:50:04.887803 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.887458    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a3d1c5cd-c0aa-446c-91a2-1df64839bf16-config\") pod \"service-ca-operator-cf7c776b7-t4gdk\" (UID: \"a3d1c5cd-c0aa-446c-91a2-1df64839bf16\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-t4gdk"
Sep 11 11:50:04.887803 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.887505    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrltp\" (UniqueName: \"kubernetes.io/projected/a3d1c5cd-c0aa-446c-91a2-1df64839bf16-kube-api-access-xrltp\") pod \"service-ca-operator-cf7c776b7-t4gdk\" (UID: \"a3d1c5cd-c0aa-446c-91a2-1df64839bf16\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-t4gdk"
Sep 11 11:50:04.888513 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.888482    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a3d1c5cd-c0aa-446c-91a2-1df64839bf16-config\") pod \"service-ca-operator-cf7c776b7-t4gdk\" (UID: \"a3d1c5cd-c0aa-446c-91a2-1df64839bf16\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-t4gdk"
Sep 11 11:50:04.890014 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.889993    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a3d1c5cd-c0aa-446c-91a2-1df64839bf16-serving-cert\") pod \"service-ca-operator-cf7c776b7-t4gdk\" (UID: \"a3d1c5cd-c0aa-446c-91a2-1df64839bf16\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-t4gdk"
Sep 11 11:50:04.899823 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.899800    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrltp\" (UniqueName: \"kubernetes.io/projected/a3d1c5cd-c0aa-446c-91a2-1df64839bf16-kube-api-access-xrltp\") pod \"service-ca-operator-cf7c776b7-t4gdk\" (UID: \"a3d1c5cd-c0aa-446c-91a2-1df64839bf16\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-t4gdk"
Sep 11 11:50:04.900056 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.900037    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8c6sp\" (UniqueName: \"kubernetes.io/projected/26a684aa-1e23-44b8-a876-7ee9bbf2709c-kube-api-access-8c6sp\") pod \"network-check-source-6f8b768455-twxgv\" (UID: \"26a684aa-1e23-44b8-a876-7ee9bbf2709c\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-twxgv"
Sep 11 11:50:04.947792 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:04.947757    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-9l6kz"
Sep 11 11:50:05.021754 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:05.021717    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-t4gdk"
Sep 11 11:50:05.035847 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:05.035794    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-twxgv"
Sep 11 11:50:05.095202 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:05.095165    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-ch96n"]
Sep 11 11:50:05.222645 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:05.222512    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-9l6kz"]
Sep 11 11:50:05.225023 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:50:05.224991    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f0c67d9_ee3f_4f68_9e53_2b6e8b238d5e.slice/crio-8bba39e2103893276e45dd1edec698bfb094ff4420fd2c7165d48fcc77b39022 WatchSource:0}: Error finding container 8bba39e2103893276e45dd1edec698bfb094ff4420fd2c7165d48fcc77b39022: Status 404 returned error can't find the container with id 8bba39e2103893276e45dd1edec698bfb094ff4420fd2c7165d48fcc77b39022
Sep 11 11:50:05.290121 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:05.290035    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e8b4926-8dc4-4622-9882-8d039646c1c0-service-ca-bundle\") pod \"router-default-7b5c4b4df6-24w55\" (UID: \"6e8b4926-8dc4-4622-9882-8d039646c1c0\") " pod="openshift-ingress/router-default-7b5c4b4df6-24w55"
Sep 11 11:50:05.290121 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:05.290095    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6e8b4926-8dc4-4622-9882-8d039646c1c0-metrics-certs\") pod \"router-default-7b5c4b4df6-24w55\" (UID: \"6e8b4926-8dc4-4622-9882-8d039646c1c0\") " pod="openshift-ingress/router-default-7b5c4b4df6-24w55"
Sep 11 11:50:05.290297 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:50:05.290211    2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 11 11:50:05.290297 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:50:05.290255    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6e8b4926-8dc4-4622-9882-8d039646c1c0-service-ca-bundle podName:6e8b4926-8dc4-4622-9882-8d039646c1c0 nodeName:}" failed. No retries permitted until 2026-09-11 11:50:06.2902338 +0000 UTC m=+110.187675540 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6e8b4926-8dc4-4622-9882-8d039646c1c0-service-ca-bundle") pod "router-default-7b5c4b4df6-24w55" (UID: "6e8b4926-8dc4-4622-9882-8d039646c1c0") : configmap references non-existent config key: service-ca.crt
Sep 11 11:50:05.290297 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:50:05.290272    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e8b4926-8dc4-4622-9882-8d039646c1c0-metrics-certs podName:6e8b4926-8dc4-4622-9882-8d039646c1c0 nodeName:}" failed. No retries permitted until 2026-09-11 11:50:06.290265275 +0000 UTC m=+110.187706997 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6e8b4926-8dc4-4622-9882-8d039646c1c0-metrics-certs") pod "router-default-7b5c4b4df6-24w55" (UID: "6e8b4926-8dc4-4622-9882-8d039646c1c0") : secret "router-metrics-certs-default" not found
Sep 11 11:50:05.294257 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:05.294227    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-t4gdk"]
Sep 11 11:50:05.297818 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:50:05.297790    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3d1c5cd_c0aa_446c_91a2_1df64839bf16.slice/crio-c245cc755c9073df16a94e0269c6b24aab1ef2d861ecff051075493919114fc3 WatchSource:0}: Error finding container c245cc755c9073df16a94e0269c6b24aab1ef2d861ecff051075493919114fc3: Status 404 returned error can't find the container with id c245cc755c9073df16a94e0269c6b24aab1ef2d861ecff051075493919114fc3
Sep 11 11:50:05.315771 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:05.315742    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-twxgv"]
Sep 11 11:50:05.320266 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:50:05.320241    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26a684aa_1e23_44b8_a876_7ee9bbf2709c.slice/crio-c623de0cf3dd1c4a5ea228bc6598903a7a4f45ff3b7b4531c9b3b29c3fd9f55a WatchSource:0}: Error finding container c623de0cf3dd1c4a5ea228bc6598903a7a4f45ff3b7b4531c9b3b29c3fd9f55a: Status 404 returned error can't find the container with id c623de0cf3dd1c4a5ea228bc6598903a7a4f45ff3b7b4531c9b3b29c3fd9f55a
Sep 11 11:50:05.942614 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:05.942580    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-9l6kz" event={"ID":"7f0c67d9-ee3f-4f68-9e53-2b6e8b238d5e","Type":"ContainerStarted","Data":"8bba39e2103893276e45dd1edec698bfb094ff4420fd2c7165d48fcc77b39022"}
Sep 11 11:50:05.944093 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:05.944061    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-twxgv" event={"ID":"26a684aa-1e23-44b8-a876-7ee9bbf2709c","Type":"ContainerStarted","Data":"f0e68801cad3db1c49d7698721240b31d9eb5bf6bca781990de18f348a25473f"}
Sep 11 11:50:05.944093 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:05.944097    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-twxgv" event={"ID":"26a684aa-1e23-44b8-a876-7ee9bbf2709c","Type":"ContainerStarted","Data":"c623de0cf3dd1c4a5ea228bc6598903a7a4f45ff3b7b4531c9b3b29c3fd9f55a"}
Sep 11 11:50:05.945167 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:05.945146    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-t4gdk" event={"ID":"a3d1c5cd-c0aa-446c-91a2-1df64839bf16","Type":"ContainerStarted","Data":"c245cc755c9073df16a94e0269c6b24aab1ef2d861ecff051075493919114fc3"}
Sep 11 11:50:05.946237 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:05.946220    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-ch96n" event={"ID":"1f0f9fa5-cbd1-468b-ae81-82b48c101d37","Type":"ContainerStarted","Data":"1cfcb51f4b86998d8636b5b23be4580baa14189621c458464284b3bbc8a6471a"}
Sep 11 11:50:05.962297 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:05.962249    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f8b768455-twxgv" podStartSLOduration=1.962235389 podStartE2EDuration="1.962235389s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:50:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:50:05.962233342 +0000 UTC m=+109.859675086" watchObservedRunningTime="2026-09-11 11:50:05.962235389 +0000 UTC m=+109.859677131"
Sep 11 11:50:06.297652 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:06.297555    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6e8b4926-8dc4-4622-9882-8d039646c1c0-metrics-certs\") pod \"router-default-7b5c4b4df6-24w55\" (UID: \"6e8b4926-8dc4-4622-9882-8d039646c1c0\") " pod="openshift-ingress/router-default-7b5c4b4df6-24w55"
Sep 11 11:50:06.297822 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:06.297661    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e8b4926-8dc4-4622-9882-8d039646c1c0-service-ca-bundle\") pod \"router-default-7b5c4b4df6-24w55\" (UID: \"6e8b4926-8dc4-4622-9882-8d039646c1c0\") " pod="openshift-ingress/router-default-7b5c4b4df6-24w55"
Sep 11 11:50:06.297822 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:50:06.297729    2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 11 11:50:06.297822 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:50:06.297811    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e8b4926-8dc4-4622-9882-8d039646c1c0-metrics-certs podName:6e8b4926-8dc4-4622-9882-8d039646c1c0 nodeName:}" failed. No retries permitted until 2026-09-11 11:50:08.297789285 +0000 UTC m=+112.195231028 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6e8b4926-8dc4-4622-9882-8d039646c1c0-metrics-certs") pod "router-default-7b5c4b4df6-24w55" (UID: "6e8b4926-8dc4-4622-9882-8d039646c1c0") : secret "router-metrics-certs-default" not found
Sep 11 11:50:06.297980 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:50:06.297847    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6e8b4926-8dc4-4622-9882-8d039646c1c0-service-ca-bundle podName:6e8b4926-8dc4-4622-9882-8d039646c1c0 nodeName:}" failed. No retries permitted until 2026-09-11 11:50:08.297825883 +0000 UTC m=+112.195267611 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6e8b4926-8dc4-4622-9882-8d039646c1c0-service-ca-bundle") pod "router-default-7b5c4b4df6-24w55" (UID: "6e8b4926-8dc4-4622-9882-8d039646c1c0") : configmap references non-existent config key: service-ca.crt
Sep 11 11:50:08.314933 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:08.314893    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6e8b4926-8dc4-4622-9882-8d039646c1c0-metrics-certs\") pod \"router-default-7b5c4b4df6-24w55\" (UID: \"6e8b4926-8dc4-4622-9882-8d039646c1c0\") " pod="openshift-ingress/router-default-7b5c4b4df6-24w55"
Sep 11 11:50:08.315345 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:08.314968    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e8b4926-8dc4-4622-9882-8d039646c1c0-service-ca-bundle\") pod \"router-default-7b5c4b4df6-24w55\" (UID: \"6e8b4926-8dc4-4622-9882-8d039646c1c0\") " pod="openshift-ingress/router-default-7b5c4b4df6-24w55"
Sep 11 11:50:08.315345 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:50:08.315074    2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 11 11:50:08.315345 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:50:08.315109    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6e8b4926-8dc4-4622-9882-8d039646c1c0-service-ca-bundle podName:6e8b4926-8dc4-4622-9882-8d039646c1c0 nodeName:}" failed. No retries permitted until 2026-09-11 11:50:12.315096039 +0000 UTC m=+116.212537761 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6e8b4926-8dc4-4622-9882-8d039646c1c0-service-ca-bundle") pod "router-default-7b5c4b4df6-24w55" (UID: "6e8b4926-8dc4-4622-9882-8d039646c1c0") : configmap references non-existent config key: service-ca.crt
Sep 11 11:50:08.315345 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:50:08.315143    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e8b4926-8dc4-4622-9882-8d039646c1c0-metrics-certs podName:6e8b4926-8dc4-4622-9882-8d039646c1c0 nodeName:}" failed. No retries permitted until 2026-09-11 11:50:12.31512583 +0000 UTC m=+116.212567557 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6e8b4926-8dc4-4622-9882-8d039646c1c0-metrics-certs") pod "router-default-7b5c4b4df6-24w55" (UID: "6e8b4926-8dc4-4622-9882-8d039646c1c0") : secret "router-metrics-certs-default" not found
Sep 11 11:50:08.690214 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:08.690183    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:08.958802 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:08.958705    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-t4gdk" event={"ID":"a3d1c5cd-c0aa-446c-91a2-1df64839bf16","Type":"ContainerStarted","Data":"a3b51f513bf6e045c717fc762eb277e4486701d7c340984b73bcc54c2e4897c9"}
Sep 11 11:50:08.960542 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:08.960520    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-ch96n_1f0f9fa5-cbd1-468b-ae81-82b48c101d37/console-operator/0.log"
Sep 11 11:50:08.960649 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:08.960557    2579 generic.go:356] "Generic (PLEG): container finished" podID="1f0f9fa5-cbd1-468b-ae81-82b48c101d37" containerID="f856d98d2ed34c21cbdeb05eb786497565329176960143091dd2ee10b903f31e" exitCode=255
Sep 11 11:50:08.960700 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:08.960656    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-ch96n" event={"ID":"1f0f9fa5-cbd1-468b-ae81-82b48c101d37","Type":"ContainerDied","Data":"f856d98d2ed34c21cbdeb05eb786497565329176960143091dd2ee10b903f31e"}
Sep 11 11:50:08.960863 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:08.960845    2579 scope.go:117] "RemoveContainer" containerID="f856d98d2ed34c21cbdeb05eb786497565329176960143091dd2ee10b903f31e"
Sep 11 11:50:08.962146 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:08.962114    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-9l6kz" event={"ID":"7f0c67d9-ee3f-4f68-9e53-2b6e8b238d5e","Type":"ContainerStarted","Data":"ff35c35e1e1f0447c126f60b431f87667ec37051989f76d7e368885a596cd3f7"}
Sep 11 11:50:09.012728 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:09.012671    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-t4gdk" podStartSLOduration=1.65728305 podStartE2EDuration="5.012647085s" totalImagesPullingTime="3.355364035s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:50:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:50:08.985273431 +0000 UTC m=+112.882715175" watchObservedRunningTime="2026-09-11 11:50:09.012647085 +0000 UTC m=+112.910088824"
Sep 11 11:50:09.040869 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:09.040819    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5844b856db-9l6kz" podStartSLOduration=1.619371338 podStartE2EDuration="5.040803154s" totalImagesPullingTime="3.421431816s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:50:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:50:09.039473695 +0000 UTC m=+112.936915444" watchObservedRunningTime="2026-09-11 11:50:09.040803154 +0000 UTC m=+112.938244899"
Sep 11 11:50:09.336092 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:09.336007    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:09.357790 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:09.357753    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-6dtzr"]
Sep 11 11:50:09.359659 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:09.359641    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-6dtzr"
Sep 11 11:50:09.365023 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:09.364995    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-v4xnc\""
Sep 11 11:50:09.365023 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:09.365016    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 11 11:50:09.365225 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:09.365000    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 11 11:50:09.379508 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:09.379481    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-6dtzr"]
Sep 11 11:50:09.424017 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:09.423978    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9z4fm\" (UniqueName: \"kubernetes.io/projected/b6664260-08f2-4521-93a3-968b672e69da-kube-api-access-9z4fm\") pod \"migrator-8b4d697c5-6dtzr\" (UID: \"b6664260-08f2-4521-93a3-968b672e69da\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-6dtzr"
Sep 11 11:50:09.436257 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:09.436230    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:09.525093 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:09.525050    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9z4fm\" (UniqueName: \"kubernetes.io/projected/b6664260-08f2-4521-93a3-968b672e69da-kube-api-access-9z4fm\") pod \"migrator-8b4d697c5-6dtzr\" (UID: \"b6664260-08f2-4521-93a3-968b672e69da\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-6dtzr"
Sep 11 11:50:09.538485 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:09.538447    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:09.540269 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:09.540250    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9z4fm\" (UniqueName: \"kubernetes.io/projected/b6664260-08f2-4521-93a3-968b672e69da-kube-api-access-9z4fm\") pod \"migrator-8b4d697c5-6dtzr\" (UID: \"b6664260-08f2-4521-93a3-968b672e69da\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-6dtzr"
Sep 11 11:50:09.668768 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:09.668733    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-6dtzr"
Sep 11 11:50:09.738905 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:09.738833    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-kmg7h"]
Sep 11 11:50:09.744000 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:09.743958    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-kmg7h"
Sep 11 11:50:09.749291 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:09.749268    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 11 11:50:09.749893 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:09.749868    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-5tl7g\""
Sep 11 11:50:09.750039 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:09.749868    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 11 11:50:09.759574 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:09.759494    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-kmg7h"]
Sep 11 11:50:09.776164 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:09.776136    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:09.827662 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:09.827628    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhqpr\" (UniqueName: \"kubernetes.io/projected/4b394c7d-887e-48b3-bb4b-9dea65f2c51b-kube-api-access-qhqpr\") pod \"insights-runtime-extractor-kmg7h\" (UID: \"4b394c7d-887e-48b3-bb4b-9dea65f2c51b\") " pod="openshift-insights/insights-runtime-extractor-kmg7h"
Sep 11 11:50:09.827848 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:09.827674    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4b394c7d-887e-48b3-bb4b-9dea65f2c51b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kmg7h\" (UID: \"4b394c7d-887e-48b3-bb4b-9dea65f2c51b\") " pod="openshift-insights/insights-runtime-extractor-kmg7h"
Sep 11 11:50:09.827848 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:09.827769    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4b394c7d-887e-48b3-bb4b-9dea65f2c51b-data-volume\") pod \"insights-runtime-extractor-kmg7h\" (UID: \"4b394c7d-887e-48b3-bb4b-9dea65f2c51b\") " pod="openshift-insights/insights-runtime-extractor-kmg7h"
Sep 11 11:50:09.827848 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:09.827798    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4b394c7d-887e-48b3-bb4b-9dea65f2c51b-crio-socket\") pod \"insights-runtime-extractor-kmg7h\" (UID: \"4b394c7d-887e-48b3-bb4b-9dea65f2c51b\") " pod="openshift-insights/insights-runtime-extractor-kmg7h"
Sep 11 11:50:09.827848 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:09.827815    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4b394c7d-887e-48b3-bb4b-9dea65f2c51b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kmg7h\" (UID: \"4b394c7d-887e-48b3-bb4b-9dea65f2c51b\") " pod="openshift-insights/insights-runtime-extractor-kmg7h"
Sep 11 11:50:09.880426 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:09.880351    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:09.928670 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:09.928597    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4b394c7d-887e-48b3-bb4b-9dea65f2c51b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kmg7h\" (UID: \"4b394c7d-887e-48b3-bb4b-9dea65f2c51b\") " pod="openshift-insights/insights-runtime-extractor-kmg7h"
Sep 11 11:50:09.928670 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:09.928658    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4b394c7d-887e-48b3-bb4b-9dea65f2c51b-data-volume\") pod \"insights-runtime-extractor-kmg7h\" (UID: \"4b394c7d-887e-48b3-bb4b-9dea65f2c51b\") " pod="openshift-insights/insights-runtime-extractor-kmg7h"
Sep 11 11:50:09.928861 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:09.928676    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4b394c7d-887e-48b3-bb4b-9dea65f2c51b-crio-socket\") pod \"insights-runtime-extractor-kmg7h\" (UID: \"4b394c7d-887e-48b3-bb4b-9dea65f2c51b\") " pod="openshift-insights/insights-runtime-extractor-kmg7h"
Sep 11 11:50:09.928861 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:09.928692    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4b394c7d-887e-48b3-bb4b-9dea65f2c51b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kmg7h\" (UID: \"4b394c7d-887e-48b3-bb4b-9dea65f2c51b\") " pod="openshift-insights/insights-runtime-extractor-kmg7h"
Sep 11 11:50:09.928861 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:09.928734    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhqpr\" (UniqueName: \"kubernetes.io/projected/4b394c7d-887e-48b3-bb4b-9dea65f2c51b-kube-api-access-qhqpr\") pod \"insights-runtime-extractor-kmg7h\" (UID: \"4b394c7d-887e-48b3-bb4b-9dea65f2c51b\") " pod="openshift-insights/insights-runtime-extractor-kmg7h"
Sep 11 11:50:09.928861 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:50:09.928756    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 11:50:09.928861 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:50:09.928816    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 11:50:09.928861 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:09.928822    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4b394c7d-887e-48b3-bb4b-9dea65f2c51b-crio-socket\") pod \"insights-runtime-extractor-kmg7h\" (UID: \"4b394c7d-887e-48b3-bb4b-9dea65f2c51b\") " pod="openshift-insights/insights-runtime-extractor-kmg7h"
Sep 11 11:50:09.928861 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:50:09.928829    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b394c7d-887e-48b3-bb4b-9dea65f2c51b-insights-runtime-extractor-tls podName:4b394c7d-887e-48b3-bb4b-9dea65f2c51b nodeName:}" failed. No retries permitted until 2026-09-11 11:50:10.428809386 +0000 UTC m=+114.326251109 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4b394c7d-887e-48b3-bb4b-9dea65f2c51b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kmg7h" (UID: "4b394c7d-887e-48b3-bb4b-9dea65f2c51b") : secret "insights-runtime-extractor-tls" not found
Sep 11 11:50:09.929106 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:50:09.928882    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4b394c7d-887e-48b3-bb4b-9dea65f2c51b-kube-rbac-proxy-cm podName:4b394c7d-887e-48b3-bb4b-9dea65f2c51b nodeName:}" failed. No retries permitted until 2026-09-11 11:50:10.428865266 +0000 UTC m=+114.326307004 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4b394c7d-887e-48b3-bb4b-9dea65f2c51b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kmg7h" (UID: "4b394c7d-887e-48b3-bb4b-9dea65f2c51b") : configmap "kube-rbac-proxy" not found
Sep 11 11:50:09.929106 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:09.929026    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4b394c7d-887e-48b3-bb4b-9dea65f2c51b-data-volume\") pod \"insights-runtime-extractor-kmg7h\" (UID: \"4b394c7d-887e-48b3-bb4b-9dea65f2c51b\") " pod="openshift-insights/insights-runtime-extractor-kmg7h"
Sep 11 11:50:09.940154 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:09.940115    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhqpr\" (UniqueName: \"kubernetes.io/projected/4b394c7d-887e-48b3-bb4b-9dea65f2c51b-kube-api-access-qhqpr\") pod \"insights-runtime-extractor-kmg7h\" (UID: \"4b394c7d-887e-48b3-bb4b-9dea65f2c51b\") " pod="openshift-insights/insights-runtime-extractor-kmg7h"
Sep 11 11:50:09.947516 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:09.947486    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-6dtzr"]
Sep 11 11:50:09.950471 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:50:09.950448    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6664260_08f2_4521_93a3_968b672e69da.slice/crio-c9c1676021f869581186c842f26b750d64d28491cdb95f2ca011b2e2468f24de WatchSource:0}: Error finding container c9c1676021f869581186c842f26b750d64d28491cdb95f2ca011b2e2468f24de: Status 404 returned error can't find the container with id c9c1676021f869581186c842f26b750d64d28491cdb95f2ca011b2e2468f24de
Sep 11 11:50:09.970051 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:09.970023    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-ch96n_1f0f9fa5-cbd1-468b-ae81-82b48c101d37/console-operator/1.log"
Sep 11 11:50:09.970494 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:09.970474    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-ch96n_1f0f9fa5-cbd1-468b-ae81-82b48c101d37/console-operator/0.log"
Sep 11 11:50:09.970625 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:09.970517    2579 generic.go:356] "Generic (PLEG): container finished" podID="1f0f9fa5-cbd1-468b-ae81-82b48c101d37" containerID="6acc3f4efe1592cd41686214d2bf427d6110ae4f7d2ea8486a76898851226d3c" exitCode=255
Sep 11 11:50:09.970625 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:09.970597    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-ch96n" event={"ID":"1f0f9fa5-cbd1-468b-ae81-82b48c101d37","Type":"ContainerDied","Data":"6acc3f4efe1592cd41686214d2bf427d6110ae4f7d2ea8486a76898851226d3c"}
Sep 11 11:50:09.970740 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:09.970693    2579 scope.go:117] "RemoveContainer" containerID="f856d98d2ed34c21cbdeb05eb786497565329176960143091dd2ee10b903f31e"
Sep 11 11:50:09.970954 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:09.970933    2579 scope.go:117] "RemoveContainer" containerID="6acc3f4efe1592cd41686214d2bf427d6110ae4f7d2ea8486a76898851226d3c"
Sep 11 11:50:09.971177 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:50:09.971155    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-ch96n_openshift-console-operator(1f0f9fa5-cbd1-468b-ae81-82b48c101d37)\"" pod="openshift-console-operator/console-operator-696884fb9c-ch96n" podUID="1f0f9fa5-cbd1-468b-ae81-82b48c101d37"
Sep 11 11:50:09.971923 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:09.971884    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-6dtzr" event={"ID":"b6664260-08f2-4521-93a3-968b672e69da","Type":"ContainerStarted","Data":"c9c1676021f869581186c842f26b750d64d28491cdb95f2ca011b2e2468f24de"}
Sep 11 11:50:10.433658 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:10.433622    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4b394c7d-887e-48b3-bb4b-9dea65f2c51b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kmg7h\" (UID: \"4b394c7d-887e-48b3-bb4b-9dea65f2c51b\") " pod="openshift-insights/insights-runtime-extractor-kmg7h"
Sep 11 11:50:10.434121 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:10.433688    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4b394c7d-887e-48b3-bb4b-9dea65f2c51b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kmg7h\" (UID: \"4b394c7d-887e-48b3-bb4b-9dea65f2c51b\") " pod="openshift-insights/insights-runtime-extractor-kmg7h"
Sep 11 11:50:10.434121 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:50:10.433784    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 11:50:10.434121 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:50:10.433808    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 11:50:10.434121 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:50:10.433836    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4b394c7d-887e-48b3-bb4b-9dea65f2c51b-kube-rbac-proxy-cm podName:4b394c7d-887e-48b3-bb4b-9dea65f2c51b nodeName:}" failed. No retries permitted until 2026-09-11 11:50:11.43382364 +0000 UTC m=+115.331265363 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4b394c7d-887e-48b3-bb4b-9dea65f2c51b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kmg7h" (UID: "4b394c7d-887e-48b3-bb4b-9dea65f2c51b") : configmap "kube-rbac-proxy" not found
Sep 11 11:50:10.434121 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:50:10.433885    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b394c7d-887e-48b3-bb4b-9dea65f2c51b-insights-runtime-extractor-tls podName:4b394c7d-887e-48b3-bb4b-9dea65f2c51b nodeName:}" failed. No retries permitted until 2026-09-11 11:50:11.433865092 +0000 UTC m=+115.331306816 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4b394c7d-887e-48b3-bb4b-9dea65f2c51b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kmg7h" (UID: "4b394c7d-887e-48b3-bb4b-9dea65f2c51b") : secret "insights-runtime-extractor-tls" not found
Sep 11 11:50:10.668823 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:10.668780    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:10.735238 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:10.735148    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:10.908887 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:10.908852    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:10.976005 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:10.975979    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-ch96n_1f0f9fa5-cbd1-468b-ae81-82b48c101d37/console-operator/1.log"
Sep 11 11:50:10.976313 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:10.976298    2579 scope.go:117] "RemoveContainer" containerID="6acc3f4efe1592cd41686214d2bf427d6110ae4f7d2ea8486a76898851226d3c"
Sep 11 11:50:10.976480 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:50:10.976462    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-ch96n_openshift-console-operator(1f0f9fa5-cbd1-468b-ae81-82b48c101d37)\"" pod="openshift-console-operator/console-operator-696884fb9c-ch96n" podUID="1f0f9fa5-cbd1-468b-ae81-82b48c101d37"
Sep 11 11:50:10.999565 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:10.999471    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:11.380436 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:11.380365    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:11.444173 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:11.444134    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4b394c7d-887e-48b3-bb4b-9dea65f2c51b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kmg7h\" (UID: \"4b394c7d-887e-48b3-bb4b-9dea65f2c51b\") " pod="openshift-insights/insights-runtime-extractor-kmg7h"
Sep 11 11:50:11.444544 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:11.444212    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4b394c7d-887e-48b3-bb4b-9dea65f2c51b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kmg7h\" (UID: \"4b394c7d-887e-48b3-bb4b-9dea65f2c51b\") " pod="openshift-insights/insights-runtime-extractor-kmg7h"
Sep 11 11:50:11.444544 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:50:11.444262    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 11:50:11.444544 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:50:11.444327    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 11:50:11.444544 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:50:11.444328    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4b394c7d-887e-48b3-bb4b-9dea65f2c51b-kube-rbac-proxy-cm podName:4b394c7d-887e-48b3-bb4b-9dea65f2c51b nodeName:}" failed. No retries permitted until 2026-09-11 11:50:13.444311469 +0000 UTC m=+117.341753208 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4b394c7d-887e-48b3-bb4b-9dea65f2c51b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kmg7h" (UID: "4b394c7d-887e-48b3-bb4b-9dea65f2c51b") : configmap "kube-rbac-proxy" not found
Sep 11 11:50:11.444544 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:50:11.444379    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b394c7d-887e-48b3-bb4b-9dea65f2c51b-insights-runtime-extractor-tls podName:4b394c7d-887e-48b3-bb4b-9dea65f2c51b nodeName:}" failed. No retries permitted until 2026-09-11 11:50:13.444364284 +0000 UTC m=+117.341806011 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4b394c7d-887e-48b3-bb4b-9dea65f2c51b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kmg7h" (UID: "4b394c7d-887e-48b3-bb4b-9dea65f2c51b") : secret "insights-runtime-extractor-tls" not found
Sep 11 11:50:11.481660 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:11.481617    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:12.264237 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:12.264205    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-2jqbl_cdaded97-b0ac-4688-8721-813fc905c103/dns-node-resolver/0.log"
Sep 11 11:50:12.352524 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:12.352483    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6e8b4926-8dc4-4622-9882-8d039646c1c0-metrics-certs\") pod \"router-default-7b5c4b4df6-24w55\" (UID: \"6e8b4926-8dc4-4622-9882-8d039646c1c0\") " pod="openshift-ingress/router-default-7b5c4b4df6-24w55"
Sep 11 11:50:12.352729 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:12.352571    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e8b4926-8dc4-4622-9882-8d039646c1c0-service-ca-bundle\") pod \"router-default-7b5c4b4df6-24w55\" (UID: \"6e8b4926-8dc4-4622-9882-8d039646c1c0\") " pod="openshift-ingress/router-default-7b5c4b4df6-24w55"
Sep 11 11:50:12.352729 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:50:12.352603    2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 11 11:50:12.352729 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:50:12.352665    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e8b4926-8dc4-4622-9882-8d039646c1c0-metrics-certs podName:6e8b4926-8dc4-4622-9882-8d039646c1c0 nodeName:}" failed. No retries permitted until 2026-09-11 11:50:20.352646749 +0000 UTC m=+124.250088488 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6e8b4926-8dc4-4622-9882-8d039646c1c0-metrics-certs") pod "router-default-7b5c4b4df6-24w55" (UID: "6e8b4926-8dc4-4622-9882-8d039646c1c0") : secret "router-metrics-certs-default" not found
Sep 11 11:50:12.352926 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:50:12.352746    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6e8b4926-8dc4-4622-9882-8d039646c1c0-service-ca-bundle podName:6e8b4926-8dc4-4622-9882-8d039646c1c0 nodeName:}" failed. No retries permitted until 2026-09-11 11:50:20.352725753 +0000 UTC m=+124.250167489 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6e8b4926-8dc4-4622-9882-8d039646c1c0-service-ca-bundle") pod "router-default-7b5c4b4df6-24w55" (UID: "6e8b4926-8dc4-4622-9882-8d039646c1c0") : configmap references non-existent config key: service-ca.crt
Sep 11 11:50:12.985546 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:12.985506    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-6dtzr" event={"ID":"b6664260-08f2-4521-93a3-968b672e69da","Type":"ContainerStarted","Data":"5888dd74400febde8607e2873960b8b3b20f819879c1623612bca2bb7785d890"}
Sep 11 11:50:12.985993 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:12.985574    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-6dtzr" event={"ID":"b6664260-08f2-4521-93a3-968b672e69da","Type":"ContainerStarted","Data":"7f99c52dfed6e5dbbdc7fab0265767528d53fc158a6912edc60890cd86e91d5e"}
Sep 11 11:50:13.007320 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:13.007264    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-6dtzr" podStartSLOduration=1.364855388 podStartE2EDuration="4.00724669s" totalImagesPullingTime="2.642391302s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:50:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:50:13.006611459 +0000 UTC m=+116.904053204" watchObservedRunningTime="2026-09-11 11:50:13.00724669 +0000 UTC m=+116.904688435"
Sep 11 11:50:13.035084 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:13.035051    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:13.136127 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:13.136087    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:13.462271 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:13.462233    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4b394c7d-887e-48b3-bb4b-9dea65f2c51b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kmg7h\" (UID: \"4b394c7d-887e-48b3-bb4b-9dea65f2c51b\") " pod="openshift-insights/insights-runtime-extractor-kmg7h"
Sep 11 11:50:13.462474 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:13.462298    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4b394c7d-887e-48b3-bb4b-9dea65f2c51b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kmg7h\" (UID: \"4b394c7d-887e-48b3-bb4b-9dea65f2c51b\") " pod="openshift-insights/insights-runtime-extractor-kmg7h"
Sep 11 11:50:13.462474 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:50:13.462381    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 11:50:13.462474 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:50:13.462387    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 11:50:13.462474 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:50:13.462461    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4b394c7d-887e-48b3-bb4b-9dea65f2c51b-kube-rbac-proxy-cm podName:4b394c7d-887e-48b3-bb4b-9dea65f2c51b nodeName:}" failed. No retries permitted until 2026-09-11 11:50:17.462448948 +0000 UTC m=+121.359890671 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4b394c7d-887e-48b3-bb4b-9dea65f2c51b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kmg7h" (UID: "4b394c7d-887e-48b3-bb4b-9dea65f2c51b") : configmap "kube-rbac-proxy" not found
Sep 11 11:50:13.462474 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:50:13.462475    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b394c7d-887e-48b3-bb4b-9dea65f2c51b-insights-runtime-extractor-tls podName:4b394c7d-887e-48b3-bb4b-9dea65f2c51b nodeName:}" failed. No retries permitted until 2026-09-11 11:50:17.462468287 +0000 UTC m=+121.359910010 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4b394c7d-887e-48b3-bb4b-9dea65f2c51b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kmg7h" (UID: "4b394c7d-887e-48b3-bb4b-9dea65f2c51b") : secret "insights-runtime-extractor-tls" not found
Sep 11 11:50:13.663737 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:13.663705    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-t7cd7_0f52f2ea-a695-422d-a16c-fa3db1f93041/node-ca/0.log"
Sep 11 11:50:13.766826 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:13.766748    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-wk5n5"]
Sep 11 11:50:13.768919 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:13.768899    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-wk5n5"
Sep 11 11:50:13.771790 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:13.771765    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-gl97j\""
Sep 11 11:50:13.771790 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:13.771765    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 11 11:50:13.771932 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:13.771773    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 11 11:50:13.785320 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:13.785287    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-wk5n5"]
Sep 11 11:50:13.865942 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:13.865907    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7d184dcd-3ce1-469a-b8e3-9db0d7c09255-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-wk5n5\" (UID: \"7d184dcd-3ce1-469a-b8e3-9db0d7c09255\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-wk5n5"
Sep 11 11:50:13.866092 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:13.865961    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7d184dcd-3ce1-469a-b8e3-9db0d7c09255-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-wk5n5\" (UID: \"7d184dcd-3ce1-469a-b8e3-9db0d7c09255\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-wk5n5"
Sep 11 11:50:13.966686 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:13.966625    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7d184dcd-3ce1-469a-b8e3-9db0d7c09255-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-wk5n5\" (UID: \"7d184dcd-3ce1-469a-b8e3-9db0d7c09255\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-wk5n5"
Sep 11 11:50:13.966864 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:13.966724    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7d184dcd-3ce1-469a-b8e3-9db0d7c09255-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-wk5n5\" (UID: \"7d184dcd-3ce1-469a-b8e3-9db0d7c09255\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-wk5n5"
Sep 11 11:50:13.966864 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:50:13.966778    2579 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 11 11:50:13.966864 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:50:13.966847    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d184dcd-3ce1-469a-b8e3-9db0d7c09255-networking-console-plugin-cert podName:7d184dcd-3ce1-469a-b8e3-9db0d7c09255 nodeName:}" failed. No retries permitted until 2026-09-11 11:50:14.466829017 +0000 UTC m=+118.364270740 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7d184dcd-3ce1-469a-b8e3-9db0d7c09255-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-wk5n5" (UID: "7d184dcd-3ce1-469a-b8e3-9db0d7c09255") : secret "networking-console-plugin-cert" not found
Sep 11 11:50:13.967336 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:13.967318    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7d184dcd-3ce1-469a-b8e3-9db0d7c09255-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-wk5n5\" (UID: \"7d184dcd-3ce1-469a-b8e3-9db0d7c09255\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-wk5n5"
Sep 11 11:50:14.465779 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:14.465746    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qvss2_9bc72241-e4f1-41eb-84dc-994cccf593c2/serve-healthcheck-canary/0.log"
Sep 11 11:50:14.470978 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:14.470945    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7d184dcd-3ce1-469a-b8e3-9db0d7c09255-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-wk5n5\" (UID: \"7d184dcd-3ce1-469a-b8e3-9db0d7c09255\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-wk5n5"
Sep 11 11:50:14.471108 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:50:14.471076    2579 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 11 11:50:14.471153 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:50:14.471129    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d184dcd-3ce1-469a-b8e3-9db0d7c09255-networking-console-plugin-cert podName:7d184dcd-3ce1-469a-b8e3-9db0d7c09255 nodeName:}" failed. No retries permitted until 2026-09-11 11:50:15.47111545 +0000 UTC m=+119.368557173 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7d184dcd-3ce1-469a-b8e3-9db0d7c09255-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-wk5n5" (UID: "7d184dcd-3ce1-469a-b8e3-9db0d7c09255") : secret "networking-console-plugin-cert" not found
Sep 11 11:50:14.666927 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:14.666892    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-6dtzr_b6664260-08f2-4521-93a3-968b672e69da/migrator/0.log"
Sep 11 11:50:14.824007 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:14.823907    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-ch96n"
Sep 11 11:50:14.824007 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:14.823984    2579 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-696884fb9c-ch96n"
Sep 11 11:50:14.824353 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:14.824335    2579 scope.go:117] "RemoveContainer" containerID="6acc3f4efe1592cd41686214d2bf427d6110ae4f7d2ea8486a76898851226d3c"
Sep 11 11:50:14.824530 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:50:14.824509    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-ch96n_openshift-console-operator(1f0f9fa5-cbd1-468b-ae81-82b48c101d37)\"" pod="openshift-console-operator/console-operator-696884fb9c-ch96n" podUID="1f0f9fa5-cbd1-468b-ae81-82b48c101d37"
Sep 11 11:50:14.867090 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:14.867062    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-6dtzr_b6664260-08f2-4521-93a3-968b672e69da/graceful-termination/0.log"
Sep 11 11:50:14.993049 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:14.993019    2579 scope.go:117] "RemoveContainer" containerID="6acc3f4efe1592cd41686214d2bf427d6110ae4f7d2ea8486a76898851226d3c"
Sep 11 11:50:14.993208 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:50:14.993165    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-ch96n_openshift-console-operator(1f0f9fa5-cbd1-468b-ae81-82b48c101d37)\"" pod="openshift-console-operator/console-operator-696884fb9c-ch96n" podUID="1f0f9fa5-cbd1-468b-ae81-82b48c101d37"
Sep 11 11:50:15.018973 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:15.018944    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:15.478590 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:15.478549    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7d184dcd-3ce1-469a-b8e3-9db0d7c09255-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-wk5n5\" (UID: \"7d184dcd-3ce1-469a-b8e3-9db0d7c09255\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-wk5n5"
Sep 11 11:50:15.478983 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:50:15.478702    2579 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 11 11:50:15.478983 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:50:15.478770    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d184dcd-3ce1-469a-b8e3-9db0d7c09255-networking-console-plugin-cert podName:7d184dcd-3ce1-469a-b8e3-9db0d7c09255 nodeName:}" failed. No retries permitted until 2026-09-11 11:50:17.478754364 +0000 UTC m=+121.376196087 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7d184dcd-3ce1-469a-b8e3-9db0d7c09255-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-wk5n5" (UID: "7d184dcd-3ce1-469a-b8e3-9db0d7c09255") : secret "networking-console-plugin-cert" not found
Sep 11 11:50:16.355368 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:16.355328    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:16.525813 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:16.525773    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:16.832102 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:16.832005    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:16.935082 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:16.935050    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:17.036144 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:17.036112    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:17.125087 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:17.125054    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:17.433502 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:17.433419    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:17.495584 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:17.495551    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4b394c7d-887e-48b3-bb4b-9dea65f2c51b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kmg7h\" (UID: \"4b394c7d-887e-48b3-bb4b-9dea65f2c51b\") " pod="openshift-insights/insights-runtime-extractor-kmg7h"
Sep 11 11:50:17.495733 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:17.495589    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7d184dcd-3ce1-469a-b8e3-9db0d7c09255-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-wk5n5\" (UID: \"7d184dcd-3ce1-469a-b8e3-9db0d7c09255\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-wk5n5"
Sep 11 11:50:17.495733 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:17.495643    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4b394c7d-887e-48b3-bb4b-9dea65f2c51b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kmg7h\" (UID: \"4b394c7d-887e-48b3-bb4b-9dea65f2c51b\") " pod="openshift-insights/insights-runtime-extractor-kmg7h"
Sep 11 11:50:17.495733 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:50:17.495672    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 11:50:17.495733 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:50:17.495730    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 11:50:17.495884 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:50:17.495758    2579 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 11 11:50:17.495884 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:50:17.495760    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4b394c7d-887e-48b3-bb4b-9dea65f2c51b-kube-rbac-proxy-cm podName:4b394c7d-887e-48b3-bb4b-9dea65f2c51b nodeName:}" failed. No retries permitted until 2026-09-11 11:50:25.495739382 +0000 UTC m=+129.393181142 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4b394c7d-887e-48b3-bb4b-9dea65f2c51b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kmg7h" (UID: "4b394c7d-887e-48b3-bb4b-9dea65f2c51b") : configmap "kube-rbac-proxy" not found
Sep 11 11:50:17.495884 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:50:17.495812    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b394c7d-887e-48b3-bb4b-9dea65f2c51b-insights-runtime-extractor-tls podName:4b394c7d-887e-48b3-bb4b-9dea65f2c51b nodeName:}" failed. No retries permitted until 2026-09-11 11:50:25.495801037 +0000 UTC m=+129.393242768 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4b394c7d-887e-48b3-bb4b-9dea65f2c51b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kmg7h" (UID: "4b394c7d-887e-48b3-bb4b-9dea65f2c51b") : secret "insights-runtime-extractor-tls" not found
Sep 11 11:50:17.495884 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:50:17.495826    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d184dcd-3ce1-469a-b8e3-9db0d7c09255-networking-console-plugin-cert podName:7d184dcd-3ce1-469a-b8e3-9db0d7c09255 nodeName:}" failed. No retries permitted until 2026-09-11 11:50:21.495817159 +0000 UTC m=+125.393258883 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7d184dcd-3ce1-469a-b8e3-9db0d7c09255-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-wk5n5" (UID: "7d184dcd-3ce1-469a-b8e3-9db0d7c09255") : secret "networking-console-plugin-cert" not found
Sep 11 11:50:17.913008 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:17.912974    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:18.549718 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:18.549683    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:18.650285 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:18.650249    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:19.062622 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:19.062590    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:19.451344 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:19.451309    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:19.529501 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:19.529467    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:19.642279 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:19.642250    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:20.242856 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:20.242827    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:20.340688 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:20.340658    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:20.415069 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:20.415037    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6e8b4926-8dc4-4622-9882-8d039646c1c0-metrics-certs\") pod \"router-default-7b5c4b4df6-24w55\" (UID: \"6e8b4926-8dc4-4622-9882-8d039646c1c0\") " pod="openshift-ingress/router-default-7b5c4b4df6-24w55"
Sep 11 11:50:20.415207 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:20.415091    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e8b4926-8dc4-4622-9882-8d039646c1c0-service-ca-bundle\") pod \"router-default-7b5c4b4df6-24w55\" (UID: \"6e8b4926-8dc4-4622-9882-8d039646c1c0\") " pod="openshift-ingress/router-default-7b5c4b4df6-24w55"
Sep 11 11:50:20.415207 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:50:20.415185    2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 11 11:50:20.415274 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:50:20.415192    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6e8b4926-8dc4-4622-9882-8d039646c1c0-service-ca-bundle podName:6e8b4926-8dc4-4622-9882-8d039646c1c0 nodeName:}" failed. No retries permitted until 2026-09-11 11:50:36.415178555 +0000 UTC m=+140.312620278 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6e8b4926-8dc4-4622-9882-8d039646c1c0-service-ca-bundle") pod "router-default-7b5c4b4df6-24w55" (UID: "6e8b4926-8dc4-4622-9882-8d039646c1c0") : configmap references non-existent config key: service-ca.crt
Sep 11 11:50:20.415356 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:50:20.415341    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e8b4926-8dc4-4622-9882-8d039646c1c0-metrics-certs podName:6e8b4926-8dc4-4622-9882-8d039646c1c0 nodeName:}" failed. No retries permitted until 2026-09-11 11:50:36.415320585 +0000 UTC m=+140.312762312 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6e8b4926-8dc4-4622-9882-8d039646c1c0-metrics-certs") pod "router-default-7b5c4b4df6-24w55" (UID: "6e8b4926-8dc4-4622-9882-8d039646c1c0") : secret "router-metrics-certs-default" not found
Sep 11 11:50:20.549845 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:20.549767    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:20.844645 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:20.844615    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:21.042887 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:21.042848    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:21.439291 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:21.439251    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:21.508870 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:21.508836    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:21.523929 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:21.523898    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7d184dcd-3ce1-469a-b8e3-9db0d7c09255-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-wk5n5\" (UID: \"7d184dcd-3ce1-469a-b8e3-9db0d7c09255\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-wk5n5"
Sep 11 11:50:21.524126 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:50:21.524062    2579 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 11 11:50:21.524189 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:50:21.524140    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d184dcd-3ce1-469a-b8e3-9db0d7c09255-networking-console-plugin-cert podName:7d184dcd-3ce1-469a-b8e3-9db0d7c09255 nodeName:}" failed. No retries permitted until 2026-09-11 11:50:29.524120455 +0000 UTC m=+133.421562195 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7d184dcd-3ce1-469a-b8e3-9db0d7c09255-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-wk5n5" (UID: "7d184dcd-3ce1-469a-b8e3-9db0d7c09255") : secret "networking-console-plugin-cert" not found
Sep 11 11:50:21.639614 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:21.639580    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:22.232212 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:22.232176    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:22.446344 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:22.446310    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:22.647270 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:22.647240    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:22.747443 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:22.747411    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:22.905737 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:22.905653    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:23.143476 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:23.143445    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:23.241541 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:23.241457    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:23.836721 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:23.836687    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:24.009464 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:24.009432    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:24.430685 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:24.430650    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:24.627020 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:24.626988    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:25.033981 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:25.033948    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:25.233672 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:25.233628    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:25.423999 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:25.423966    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:25.554890 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:25.554857    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4b394c7d-887e-48b3-bb4b-9dea65f2c51b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kmg7h\" (UID: \"4b394c7d-887e-48b3-bb4b-9dea65f2c51b\") " pod="openshift-insights/insights-runtime-extractor-kmg7h"
Sep 11 11:50:25.555076 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:25.554936    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4b394c7d-887e-48b3-bb4b-9dea65f2c51b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kmg7h\" (UID: \"4b394c7d-887e-48b3-bb4b-9dea65f2c51b\") " pod="openshift-insights/insights-runtime-extractor-kmg7h"
Sep 11 11:50:25.555076 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:50:25.555036    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 11:50:25.555188 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:50:25.555097    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b394c7d-887e-48b3-bb4b-9dea65f2c51b-insights-runtime-extractor-tls podName:4b394c7d-887e-48b3-bb4b-9dea65f2c51b nodeName:}" failed. No retries permitted until 2026-09-11 11:50:41.55508389 +0000 UTC m=+145.452525612 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4b394c7d-887e-48b3-bb4b-9dea65f2c51b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kmg7h" (UID: "4b394c7d-887e-48b3-bb4b-9dea65f2c51b") : secret "insights-runtime-extractor-tls" not found
Sep 11 11:50:25.555566 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:25.555541    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4b394c7d-887e-48b3-bb4b-9dea65f2c51b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kmg7h\" (UID: \"4b394c7d-887e-48b3-bb4b-9dea65f2c51b\") " pod="openshift-insights/insights-runtime-extractor-kmg7h"
Sep 11 11:50:26.260260 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:26.260215    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa7be6d8-e93e-49af-8782-14c8080fef33-metrics-certs\") pod \"network-metrics-daemon-krxsv\" (UID: \"fa7be6d8-e93e-49af-8782-14c8080fef33\") " pod="openshift-multus/network-metrics-daemon-krxsv"
Sep 11 11:50:26.262614 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:26.262592    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa7be6d8-e93e-49af-8782-14c8080fef33-metrics-certs\") pod \"network-metrics-daemon-krxsv\" (UID: \"fa7be6d8-e93e-49af-8782-14c8080fef33\") " pod="openshift-multus/network-metrics-daemon-krxsv"
Sep 11 11:50:26.487971 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:26.487936    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5sht6\""
Sep 11 11:50:26.495492 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:26.495470    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-krxsv"
Sep 11 11:50:26.506084 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:26.506045    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:26.767487 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:26.767373    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-krxsv"]
Sep 11 11:50:26.770010 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:50:26.769982    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa7be6d8_e93e_49af_8782_14c8080fef33.slice/crio-5b1977e17b4a846596b1118d104693ab3438940134ad59de07ef4c0ccc0e2520 WatchSource:0}: Error finding container 5b1977e17b4a846596b1118d104693ab3438940134ad59de07ef4c0ccc0e2520: Status 404 returned error can't find the container with id 5b1977e17b4a846596b1118d104693ab3438940134ad59de07ef4c0ccc0e2520
Sep 11 11:50:26.917867 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:26.917838    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:27.018358 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:27.018315    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:27.028979 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:27.028947    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-krxsv" event={"ID":"fa7be6d8-e93e-49af-8782-14c8080fef33","Type":"ContainerStarted","Data":"5b1977e17b4a846596b1118d104693ab3438940134ad59de07ef4c0ccc0e2520"}
Sep 11 11:50:27.251124 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:27.251038    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:29.037546 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:29.037510    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-krxsv" event={"ID":"fa7be6d8-e93e-49af-8782-14c8080fef33","Type":"ContainerStarted","Data":"268daa35aaa36f3cf430a474091c6b2645c0fbda2946f252815a2318b92343a6"}
Sep 11 11:50:29.037976 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:29.037563    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-krxsv" event={"ID":"fa7be6d8-e93e-49af-8782-14c8080fef33","Type":"ContainerStarted","Data":"90c944063456d749909ac4ec08d902a7685d599634f622eb6700deeea67432bd"}
Sep 11 11:50:29.063231 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:29.063179    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-krxsv" podStartSLOduration=131.501725132 podStartE2EDuration="2m13.063162572s" totalImagesPullingTime="1.56143744s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:48:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:50:29.061747596 +0000 UTC m=+132.959189340" watchObservedRunningTime="2026-09-11 11:50:29.063162572 +0000 UTC m=+132.960604697"
Sep 11 11:50:29.589268 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:29.589225    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7d184dcd-3ce1-469a-b8e3-9db0d7c09255-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-wk5n5\" (UID: \"7d184dcd-3ce1-469a-b8e3-9db0d7c09255\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-wk5n5"
Sep 11 11:50:29.591599 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:29.591581    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7d184dcd-3ce1-469a-b8e3-9db0d7c09255-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-wk5n5\" (UID: \"7d184dcd-3ce1-469a-b8e3-9db0d7c09255\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-wk5n5"
Sep 11 11:50:29.681422 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:29.681376    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-gl97j\""
Sep 11 11:50:29.689218 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:29.689200    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-wk5n5"
Sep 11 11:50:29.967113 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:29.967079    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-wk5n5"]
Sep 11 11:50:29.970842 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:50:29.970819    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d184dcd_3ce1_469a_b8e3_9db0d7c09255.slice/crio-05678ae9553f38fd462cfd57e015c8f8c066e52147a7583be4ad9fd866ab5ef1 WatchSource:0}: Error finding container 05678ae9553f38fd462cfd57e015c8f8c066e52147a7583be4ad9fd866ab5ef1: Status 404 returned error can't find the container with id 05678ae9553f38fd462cfd57e015c8f8c066e52147a7583be4ad9fd866ab5ef1
Sep 11 11:50:30.041984 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:30.041942    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-wk5n5" event={"ID":"7d184dcd-3ce1-469a-b8e3-9db0d7c09255","Type":"ContainerStarted","Data":"05678ae9553f38fd462cfd57e015c8f8c066e52147a7583be4ad9fd866ab5ef1"}
Sep 11 11:50:30.256582 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:30.256490    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:30.570895 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:30.570814    2579 scope.go:117] "RemoveContainer" containerID="6acc3f4efe1592cd41686214d2bf427d6110ae4f7d2ea8486a76898851226d3c"
Sep 11 11:50:31.048689 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:31.048586    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-ch96n_1f0f9fa5-cbd1-468b-ae81-82b48c101d37/console-operator/2.log"
Sep 11 11:50:31.049339 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:31.049321    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-ch96n_1f0f9fa5-cbd1-468b-ae81-82b48c101d37/console-operator/1.log"
Sep 11 11:50:31.049467 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:31.049357    2579 generic.go:356] "Generic (PLEG): container finished" podID="1f0f9fa5-cbd1-468b-ae81-82b48c101d37" containerID="aa97bdff795abf16011a7e3f8008056ae7520c4ed835aa8ca3a6f903ad99cd38" exitCode=255
Sep 11 11:50:31.049467 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:31.049409    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-ch96n" event={"ID":"1f0f9fa5-cbd1-468b-ae81-82b48c101d37","Type":"ContainerDied","Data":"aa97bdff795abf16011a7e3f8008056ae7520c4ed835aa8ca3a6f903ad99cd38"}
Sep 11 11:50:31.049583 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:31.049490    2579 scope.go:117] "RemoveContainer" containerID="6acc3f4efe1592cd41686214d2bf427d6110ae4f7d2ea8486a76898851226d3c"
Sep 11 11:50:31.049957 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:31.049908    2579 scope.go:117] "RemoveContainer" containerID="aa97bdff795abf16011a7e3f8008056ae7520c4ed835aa8ca3a6f903ad99cd38"
Sep 11 11:50:31.050123 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:50:31.050101    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-696884fb9c-ch96n_openshift-console-operator(1f0f9fa5-cbd1-468b-ae81-82b48c101d37)\"" pod="openshift-console-operator/console-operator-696884fb9c-ch96n" podUID="1f0f9fa5-cbd1-468b-ae81-82b48c101d37"
Sep 11 11:50:32.054565 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:32.054538    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-ch96n_1f0f9fa5-cbd1-468b-ae81-82b48c101d37/console-operator/2.log"
Sep 11 11:50:32.056111 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:32.056085    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-wk5n5" event={"ID":"7d184dcd-3ce1-469a-b8e3-9db0d7c09255","Type":"ContainerStarted","Data":"bd89fa83f88b998324e60b3239da5cedacd55e74988bcd01ae9ef7e2cbd4e847"}
Sep 11 11:50:32.075433 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:32.075361    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-wk5n5" podStartSLOduration=17.755909018 podStartE2EDuration="19.075343159s" totalImagesPullingTime="1.319434141s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:50:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:50:32.074805253 +0000 UTC m=+135.972247002" watchObservedRunningTime="2026-09-11 11:50:32.075343159 +0000 UTC m=+135.972784905"
Sep 11 11:50:34.824355 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:34.824309    2579 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-696884fb9c-ch96n"
Sep 11 11:50:34.824742 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:34.824377    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-ch96n"
Sep 11 11:50:34.824742 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:34.824713    2579 scope.go:117] "RemoveContainer" containerID="aa97bdff795abf16011a7e3f8008056ae7520c4ed835aa8ca3a6f903ad99cd38"
Sep 11 11:50:34.824889 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:50:34.824870    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-696884fb9c-ch96n_openshift-console-operator(1f0f9fa5-cbd1-468b-ae81-82b48c101d37)\"" pod="openshift-console-operator/console-operator-696884fb9c-ch96n" podUID="1f0f9fa5-cbd1-468b-ae81-82b48c101d37"
Sep 11 11:50:35.971299 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:35.971265    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:36.085554 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:36.085517    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:36.185485 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:36.185451    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:36.313816 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:36.313741    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:36.448526 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:36.448491    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6e8b4926-8dc4-4622-9882-8d039646c1c0-metrics-certs\") pod \"router-default-7b5c4b4df6-24w55\" (UID: \"6e8b4926-8dc4-4622-9882-8d039646c1c0\") " pod="openshift-ingress/router-default-7b5c4b4df6-24w55"
Sep 11 11:50:36.448701 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:36.448540    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e8b4926-8dc4-4622-9882-8d039646c1c0-service-ca-bundle\") pod \"router-default-7b5c4b4df6-24w55\" (UID: \"6e8b4926-8dc4-4622-9882-8d039646c1c0\") " pod="openshift-ingress/router-default-7b5c4b4df6-24w55"
Sep 11 11:50:36.449143 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:36.449126    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e8b4926-8dc4-4622-9882-8d039646c1c0-service-ca-bundle\") pod \"router-default-7b5c4b4df6-24w55\" (UID: \"6e8b4926-8dc4-4622-9882-8d039646c1c0\") " pod="openshift-ingress/router-default-7b5c4b4df6-24w55"
Sep 11 11:50:36.450884 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:36.450863    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6e8b4926-8dc4-4622-9882-8d039646c1c0-metrics-certs\") pod \"router-default-7b5c4b4df6-24w55\" (UID: \"6e8b4926-8dc4-4622-9882-8d039646c1c0\") " pod="openshift-ingress/router-default-7b5c4b4df6-24w55"
Sep 11 11:50:36.523855 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:36.523821    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:36.603775 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:36.603745    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:36.744661 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:36.744630    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-5ld9w\""
Sep 11 11:50:36.751922 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:36.751897    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7b5c4b4df6-24w55"
Sep 11 11:50:37.028706 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:37.028610    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7b5c4b4df6-24w55"]
Sep 11 11:50:37.032775 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:50:37.032740    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e8b4926_8dc4_4622_9882_8d039646c1c0.slice/crio-f7385fbb649d780b5dce2db5ac0548ce29c01a75da63878021d04a9b4843667f WatchSource:0}: Error finding container f7385fbb649d780b5dce2db5ac0548ce29c01a75da63878021d04a9b4843667f: Status 404 returned error can't find the container with id f7385fbb649d780b5dce2db5ac0548ce29c01a75da63878021d04a9b4843667f
Sep 11 11:50:37.077847 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:37.077818    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7b5c4b4df6-24w55" event={"ID":"6e8b4926-8dc4-4622-9882-8d039646c1c0","Type":"ContainerStarted","Data":"f7385fbb649d780b5dce2db5ac0548ce29c01a75da63878021d04a9b4843667f"}
Sep 11 11:50:37.725792 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:37.725749    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:37.797430 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:37.797371    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:38.082337 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:38.082241    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7b5c4b4df6-24w55" event={"ID":"6e8b4926-8dc4-4622-9882-8d039646c1c0","Type":"ContainerStarted","Data":"84d1b4f8d90e4b0ff00bfec5c2d02252276105ab96bd3e5af51a1ccf041f918c"}
Sep 11 11:50:38.752600 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:38.752562    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-7b5c4b4df6-24w55"
Sep 11 11:50:38.755013 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:38.754990    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-7b5c4b4df6-24w55"
Sep 11 11:50:38.816574 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:38.816518    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-7b5c4b4df6-24w55" podStartSLOduration=34.816501616 podStartE2EDuration="34.816501616s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:50:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:50:38.117910206 +0000 UTC m=+142.015351952" watchObservedRunningTime="2026-09-11 11:50:38.816501616 +0000 UTC m=+142.713943362"
Sep 11 11:50:38.934506 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:38.934475    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:39.044660 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:39.044567    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:39.085456 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:39.085425    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-7b5c4b4df6-24w55"
Sep 11 11:50:39.086739 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:39.086717    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-7b5c4b4df6-24w55"
Sep 11 11:50:39.177025 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:39.176994    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-t8t5g"]
Sep 11 11:50:39.180004 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:39.179986    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-t8t5g"
Sep 11 11:50:39.183638 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:39.183614    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-svzvp\""
Sep 11 11:50:39.184061 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:39.184044    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 11 11:50:39.208520 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:39.208484    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-t8t5g"]
Sep 11 11:50:39.237538 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:39.237510    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:39.272259 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:39.272222    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/4bfe732c-eb81-4d2a-b1ae-213050a8205c-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-t8t5g\" (UID: \"4bfe732c-eb81-4d2a-b1ae-213050a8205c\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-t8t5g"
Sep 11 11:50:39.329309 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:39.329219    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:39.373322 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:39.373276    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/4bfe732c-eb81-4d2a-b1ae-213050a8205c-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-t8t5g\" (UID: \"4bfe732c-eb81-4d2a-b1ae-213050a8205c\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-t8t5g"
Sep 11 11:50:39.375864 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:39.375843    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/4bfe732c-eb81-4d2a-b1ae-213050a8205c-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-t8t5g\" (UID: \"4bfe732c-eb81-4d2a-b1ae-213050a8205c\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-t8t5g"
Sep 11 11:50:39.488735 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:39.488697    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-t8t5g"
Sep 11 11:50:39.759530 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:39.759495    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-t8t5g"]
Sep 11 11:50:39.762187 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:50:39.762164    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bfe732c_eb81_4d2a_b1ae_213050a8205c.slice/crio-552e60cd2c3d01aa2db98371badd1dcf882b12df2f687bde02f573fdae4e206d WatchSource:0}: Error finding container 552e60cd2c3d01aa2db98371badd1dcf882b12df2f687bde02f573fdae4e206d: Status 404 returned error can't find the container with id 552e60cd2c3d01aa2db98371badd1dcf882b12df2f687bde02f573fdae4e206d
Sep 11 11:50:40.089413 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:40.089315    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-t8t5g" event={"ID":"4bfe732c-eb81-4d2a-b1ae-213050a8205c","Type":"ContainerStarted","Data":"552e60cd2c3d01aa2db98371badd1dcf882b12df2f687bde02f573fdae4e206d"}
Sep 11 11:50:41.094644 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:41.094601    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-t8t5g" event={"ID":"4bfe732c-eb81-4d2a-b1ae-213050a8205c","Type":"ContainerStarted","Data":"716fae4f5e9e87bce2656aa16eb80e6c39b3c924874bd9abcf511c307f90d892"}
Sep 11 11:50:41.113693 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:41.113636    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-t8t5g" podStartSLOduration=0.95389566 podStartE2EDuration="2.113621667s" totalImagesPullingTime="1.159726007s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:50:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:50:41.111767091 +0000 UTC m=+145.009208835" watchObservedRunningTime="2026-09-11 11:50:41.113621667 +0000 UTC m=+145.011063411"
Sep 11 11:50:41.589079 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:41.589035    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4b394c7d-887e-48b3-bb4b-9dea65f2c51b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kmg7h\" (UID: \"4b394c7d-887e-48b3-bb4b-9dea65f2c51b\") " pod="openshift-insights/insights-runtime-extractor-kmg7h"
Sep 11 11:50:41.591380 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:41.591352    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4b394c7d-887e-48b3-bb4b-9dea65f2c51b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kmg7h\" (UID: \"4b394c7d-887e-48b3-bb4b-9dea65f2c51b\") " pod="openshift-insights/insights-runtime-extractor-kmg7h"
Sep 11 11:50:41.858201 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:41.858173    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-5tl7g\""
Sep 11 11:50:41.865744 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:41.865723    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-kmg7h"
Sep 11 11:50:42.097683 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:42.097652    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-t8t5g"
Sep 11 11:50:42.102291 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:42.102266    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-t8t5g"
Sep 11 11:50:42.158218 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:42.158124    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-kmg7h"]
Sep 11 11:50:42.161180 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:50:42.161141    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b394c7d_887e_48b3_bb4b_9dea65f2c51b.slice/crio-09d9f646e27881bca17ed211042c19214f4ddb90d61e8d475c7d0ca780acd1a8 WatchSource:0}: Error finding container 09d9f646e27881bca17ed211042c19214f4ddb90d61e8d475c7d0ca780acd1a8: Status 404 returned error can't find the container with id 09d9f646e27881bca17ed211042c19214f4ddb90d61e8d475c7d0ca780acd1a8
Sep 11 11:50:43.102124 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:43.102092    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kmg7h" event={"ID":"4b394c7d-887e-48b3-bb4b-9dea65f2c51b","Type":"ContainerStarted","Data":"66a9ce9a7c71b91b0298160ede235faee2d0f0201f15c4b209065cacbdc78903"}
Sep 11 11:50:43.102508 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:43.102149    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kmg7h" event={"ID":"4b394c7d-887e-48b3-bb4b-9dea65f2c51b","Type":"ContainerStarted","Data":"09d9f646e27881bca17ed211042c19214f4ddb90d61e8d475c7d0ca780acd1a8"}
Sep 11 11:50:43.167770 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:43.167734    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:43.248481 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:43.248451    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:43.392969 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:43.392881    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:44.109267 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:44.109223    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kmg7h" event={"ID":"4b394c7d-887e-48b3-bb4b-9dea65f2c51b","Type":"ContainerStarted","Data":"73016415c06f9c7c3701b6afaeeb7d78f2849326f87e288fb172656507933744"}
Sep 11 11:50:45.114734 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:45.114698    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kmg7h" event={"ID":"4b394c7d-887e-48b3-bb4b-9dea65f2c51b","Type":"ContainerStarted","Data":"0ef3ea80ff9e34e73bb77238f788c1f6ea6463a093e342e70765303e805ff725"}
Sep 11 11:50:45.135743 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:45.135691    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-kmg7h" podStartSLOduration=33.981978911 podStartE2EDuration="36.135672744s" totalImagesPullingTime="2.153693833s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 11:50:09 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:50:45.133989171 +0000 UTC m=+149.031430919" watchObservedRunningTime="2026-09-11 11:50:45.135672744 +0000 UTC m=+149.033114490"
Sep 11 11:50:45.716540 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:45.716504    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:47.375751 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:47.375717    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:47.569840 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:47.569813    2579 scope.go:117] "RemoveContainer" containerID="aa97bdff795abf16011a7e3f8008056ae7520c4ed835aa8ca3a6f903ad99cd38"
Sep 11 11:50:47.570005 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:50:47.569972    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-696884fb9c-ch96n_openshift-console-operator(1f0f9fa5-cbd1-468b-ae81-82b48c101d37)\"" pod="openshift-console-operator/console-operator-696884fb9c-ch96n" podUID="1f0f9fa5-cbd1-468b-ae81-82b48c101d37"
Sep 11 11:50:47.583597 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:47.583565    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:47.645148 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:47.645078    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-j8pjk"]
Sep 11 11:50:47.647511 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:47.647496    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-j8pjk"
Sep 11 11:50:47.656151 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:47.656134    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 11 11:50:47.659762 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:47.659748    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 11 11:50:47.660248 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:47.660236    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 11 11:50:47.660292 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:47.660270    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 11 11:50:47.661103 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:47.661091    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 11 11:50:47.661150 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:47.661109    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-shfsl\""
Sep 11 11:50:47.713943 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:47.713904    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:47.736718 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:47.736684    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a0a5db0b-9a22-4973-b161-09d4675e0843-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-j8pjk\" (UID: \"a0a5db0b-9a22-4973-b161-09d4675e0843\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-j8pjk"
Sep 11 11:50:47.736882 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:47.736736    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a0a5db0b-9a22-4973-b161-09d4675e0843-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-j8pjk\" (UID: \"a0a5db0b-9a22-4973-b161-09d4675e0843\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-j8pjk"
Sep 11 11:50:47.736882 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:47.736774    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ph4mt\" (UniqueName: \"kubernetes.io/projected/a0a5db0b-9a22-4973-b161-09d4675e0843-kube-api-access-ph4mt\") pod \"openshift-state-metrics-67dcf8c979-j8pjk\" (UID: \"a0a5db0b-9a22-4973-b161-09d4675e0843\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-j8pjk"
Sep 11 11:50:47.736882 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:47.736865    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a0a5db0b-9a22-4973-b161-09d4675e0843-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-j8pjk\" (UID: \"a0a5db0b-9a22-4973-b161-09d4675e0843\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-j8pjk"
Sep 11 11:50:47.750688 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:47.750658    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-j8pjk"]
Sep 11 11:50:47.837880 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:47.837842    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a0a5db0b-9a22-4973-b161-09d4675e0843-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-j8pjk\" (UID: \"a0a5db0b-9a22-4973-b161-09d4675e0843\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-j8pjk"
Sep 11 11:50:47.837880 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:47.837885    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ph4mt\" (UniqueName: \"kubernetes.io/projected/a0a5db0b-9a22-4973-b161-09d4675e0843-kube-api-access-ph4mt\") pod \"openshift-state-metrics-67dcf8c979-j8pjk\" (UID: \"a0a5db0b-9a22-4973-b161-09d4675e0843\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-j8pjk"
Sep 11 11:50:47.838131 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:47.837928    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a0a5db0b-9a22-4973-b161-09d4675e0843-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-j8pjk\" (UID: \"a0a5db0b-9a22-4973-b161-09d4675e0843\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-j8pjk"
Sep 11 11:50:47.838131 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:47.837964    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a0a5db0b-9a22-4973-b161-09d4675e0843-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-j8pjk\" (UID: \"a0a5db0b-9a22-4973-b161-09d4675e0843\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-j8pjk"
Sep 11 11:50:47.838597 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:47.838575    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a0a5db0b-9a22-4973-b161-09d4675e0843-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-j8pjk\" (UID: \"a0a5db0b-9a22-4973-b161-09d4675e0843\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-j8pjk"
Sep 11 11:50:47.840338 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:47.840316    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a0a5db0b-9a22-4973-b161-09d4675e0843-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-j8pjk\" (UID: \"a0a5db0b-9a22-4973-b161-09d4675e0843\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-j8pjk"
Sep 11 11:50:47.840466 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:47.840448    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a0a5db0b-9a22-4973-b161-09d4675e0843-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-j8pjk\" (UID: \"a0a5db0b-9a22-4973-b161-09d4675e0843\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-j8pjk"
Sep 11 11:50:47.852219 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:47.852191    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ph4mt\" (UniqueName: \"kubernetes.io/projected/a0a5db0b-9a22-4973-b161-09d4675e0843-kube-api-access-ph4mt\") pod \"openshift-state-metrics-67dcf8c979-j8pjk\" (UID: \"a0a5db0b-9a22-4973-b161-09d4675e0843\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-j8pjk"
Sep 11 11:50:47.860189 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:47.860163    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-p97jw"]
Sep 11 11:50:47.862297 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:47.862281    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-p97jw"
Sep 11 11:50:47.865145 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:47.865126    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 11 11:50:47.865264 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:47.865210    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 11 11:50:47.865323 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:47.865281    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 11 11:50:47.865381 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:47.865366    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-lh48f\""
Sep 11 11:50:47.878590 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:47.878561    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:47.938958 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:47.938861    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0af7a15f-31e5-4f13-85f5-0879569febcc-node-exporter-tls\") pod \"node-exporter-p97jw\" (UID: \"0af7a15f-31e5-4f13-85f5-0879569febcc\") " pod="openshift-monitoring/node-exporter-p97jw"
Sep 11 11:50:47.938958 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:47.938902    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fm9gw\" (UniqueName: \"kubernetes.io/projected/0af7a15f-31e5-4f13-85f5-0879569febcc-kube-api-access-fm9gw\") pod \"node-exporter-p97jw\" (UID: \"0af7a15f-31e5-4f13-85f5-0879569febcc\") " pod="openshift-monitoring/node-exporter-p97jw"
Sep 11 11:50:47.939143 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:47.938959    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0af7a15f-31e5-4f13-85f5-0879569febcc-sys\") pod \"node-exporter-p97jw\" (UID: \"0af7a15f-31e5-4f13-85f5-0879569febcc\") " pod="openshift-monitoring/node-exporter-p97jw"
Sep 11 11:50:47.939143 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:47.938986    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0af7a15f-31e5-4f13-85f5-0879569febcc-node-exporter-accelerators-collector-config\") pod \"node-exporter-p97jw\" (UID: \"0af7a15f-31e5-4f13-85f5-0879569febcc\") " pod="openshift-monitoring/node-exporter-p97jw"
Sep 11 11:50:47.939143 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:47.939064    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0af7a15f-31e5-4f13-85f5-0879569febcc-metrics-client-ca\") pod \"node-exporter-p97jw\" (UID: \"0af7a15f-31e5-4f13-85f5-0879569febcc\") " pod="openshift-monitoring/node-exporter-p97jw"
Sep 11 11:50:47.939143 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:47.939092    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0af7a15f-31e5-4f13-85f5-0879569febcc-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-p97jw\" (UID: \"0af7a15f-31e5-4f13-85f5-0879569febcc\") " pod="openshift-monitoring/node-exporter-p97jw"
Sep 11 11:50:47.939143 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:47.939110    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0af7a15f-31e5-4f13-85f5-0879569febcc-node-exporter-wtmp\") pod \"node-exporter-p97jw\" (UID: \"0af7a15f-31e5-4f13-85f5-0879569febcc\") " pod="openshift-monitoring/node-exporter-p97jw"
Sep 11 11:50:47.939143 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:47.939132    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0af7a15f-31e5-4f13-85f5-0879569febcc-node-exporter-textfile\") pod \"node-exporter-p97jw\" (UID: \"0af7a15f-31e5-4f13-85f5-0879569febcc\") " pod="openshift-monitoring/node-exporter-p97jw"
Sep 11 11:50:47.939319 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:47.939154    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0af7a15f-31e5-4f13-85f5-0879569febcc-root\") pod \"node-exporter-p97jw\" (UID: \"0af7a15f-31e5-4f13-85f5-0879569febcc\") " pod="openshift-monitoring/node-exporter-p97jw"
Sep 11 11:50:47.955925 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:47.955893    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-j8pjk"
Sep 11 11:50:48.021837 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.021735    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:48.040360 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.040328    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0af7a15f-31e5-4f13-85f5-0879569febcc-node-exporter-textfile\") pod \"node-exporter-p97jw\" (UID: \"0af7a15f-31e5-4f13-85f5-0879569febcc\") " pod="openshift-monitoring/node-exporter-p97jw"
Sep 11 11:50:48.040360 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.040358    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0af7a15f-31e5-4f13-85f5-0879569febcc-root\") pod \"node-exporter-p97jw\" (UID: \"0af7a15f-31e5-4f13-85f5-0879569febcc\") " pod="openshift-monitoring/node-exporter-p97jw"
Sep 11 11:50:48.040602 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.040416    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0af7a15f-31e5-4f13-85f5-0879569febcc-node-exporter-tls\") pod \"node-exporter-p97jw\" (UID: \"0af7a15f-31e5-4f13-85f5-0879569febcc\") " pod="openshift-monitoring/node-exporter-p97jw"
Sep 11 11:50:48.040602 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.040436    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fm9gw\" (UniqueName: \"kubernetes.io/projected/0af7a15f-31e5-4f13-85f5-0879569febcc-kube-api-access-fm9gw\") pod \"node-exporter-p97jw\" (UID: \"0af7a15f-31e5-4f13-85f5-0879569febcc\") " pod="openshift-monitoring/node-exporter-p97jw"
Sep 11 11:50:48.040602 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.040458    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0af7a15f-31e5-4f13-85f5-0879569febcc-sys\") pod \"node-exporter-p97jw\" (UID: \"0af7a15f-31e5-4f13-85f5-0879569febcc\") " pod="openshift-monitoring/node-exporter-p97jw"
Sep 11 11:50:48.040602 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.040476    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0af7a15f-31e5-4f13-85f5-0879569febcc-node-exporter-accelerators-collector-config\") pod \"node-exporter-p97jw\" (UID: \"0af7a15f-31e5-4f13-85f5-0879569febcc\") " pod="openshift-monitoring/node-exporter-p97jw"
Sep 11 11:50:48.040602 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.040527    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0af7a15f-31e5-4f13-85f5-0879569febcc-metrics-client-ca\") pod \"node-exporter-p97jw\" (UID: \"0af7a15f-31e5-4f13-85f5-0879569febcc\") " pod="openshift-monitoring/node-exporter-p97jw"
Sep 11 11:50:48.040602 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.040550    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0af7a15f-31e5-4f13-85f5-0879569febcc-root\") pod \"node-exporter-p97jw\" (UID: \"0af7a15f-31e5-4f13-85f5-0879569febcc\") " pod="openshift-monitoring/node-exporter-p97jw"
Sep 11 11:50:48.040602 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.040571    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0af7a15f-31e5-4f13-85f5-0879569febcc-sys\") pod \"node-exporter-p97jw\" (UID: \"0af7a15f-31e5-4f13-85f5-0879569febcc\") " pod="openshift-monitoring/node-exporter-p97jw"
Sep 11 11:50:48.040602 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.040552    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0af7a15f-31e5-4f13-85f5-0879569febcc-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-p97jw\" (UID: \"0af7a15f-31e5-4f13-85f5-0879569febcc\") " pod="openshift-monitoring/node-exporter-p97jw"
Sep 11 11:50:48.040991 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.040638    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0af7a15f-31e5-4f13-85f5-0879569febcc-node-exporter-wtmp\") pod \"node-exporter-p97jw\" (UID: \"0af7a15f-31e5-4f13-85f5-0879569febcc\") " pod="openshift-monitoring/node-exporter-p97jw"
Sep 11 11:50:48.040991 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.040708    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0af7a15f-31e5-4f13-85f5-0879569febcc-node-exporter-textfile\") pod \"node-exporter-p97jw\" (UID: \"0af7a15f-31e5-4f13-85f5-0879569febcc\") " pod="openshift-monitoring/node-exporter-p97jw"
Sep 11 11:50:48.040991 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.040786    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0af7a15f-31e5-4f13-85f5-0879569febcc-node-exporter-wtmp\") pod \"node-exporter-p97jw\" (UID: \"0af7a15f-31e5-4f13-85f5-0879569febcc\") " pod="openshift-monitoring/node-exporter-p97jw"
Sep 11 11:50:48.041093 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.041010    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0af7a15f-31e5-4f13-85f5-0879569febcc-node-exporter-accelerators-collector-config\") pod \"node-exporter-p97jw\" (UID: \"0af7a15f-31e5-4f13-85f5-0879569febcc\") " pod="openshift-monitoring/node-exporter-p97jw"
Sep 11 11:50:48.041137 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.041096    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0af7a15f-31e5-4f13-85f5-0879569febcc-metrics-client-ca\") pod \"node-exporter-p97jw\" (UID: \"0af7a15f-31e5-4f13-85f5-0879569febcc\") " pod="openshift-monitoring/node-exporter-p97jw"
Sep 11 11:50:48.043314 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.043288    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0af7a15f-31e5-4f13-85f5-0879569febcc-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-p97jw\" (UID: \"0af7a15f-31e5-4f13-85f5-0879569febcc\") " pod="openshift-monitoring/node-exporter-p97jw"
Sep 11 11:50:48.043450 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.043433    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0af7a15f-31e5-4f13-85f5-0879569febcc-node-exporter-tls\") pod \"node-exporter-p97jw\" (UID: \"0af7a15f-31e5-4f13-85f5-0879569febcc\") " pod="openshift-monitoring/node-exporter-p97jw"
Sep 11 11:50:48.063520 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.063489    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fm9gw\" (UniqueName: \"kubernetes.io/projected/0af7a15f-31e5-4f13-85f5-0879569febcc-kube-api-access-fm9gw\") pod \"node-exporter-p97jw\" (UID: \"0af7a15f-31e5-4f13-85f5-0879569febcc\") " pod="openshift-monitoring/node-exporter-p97jw"
Sep 11 11:50:48.171756 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.171720    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-p97jw"
Sep 11 11:50:48.181736 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:50:48.181697    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0af7a15f_31e5_4f13_85f5_0879569febcc.slice/crio-74ff046d89f4375fe539a74c59a9f5f9d8ee2b73e226bca41379e8b7e4cef4de WatchSource:0}: Error finding container 74ff046d89f4375fe539a74c59a9f5f9d8ee2b73e226bca41379e8b7e4cef4de: Status 404 returned error can't find the container with id 74ff046d89f4375fe539a74c59a9f5f9d8ee2b73e226bca41379e8b7e4cef4de
Sep 11 11:50:48.266472 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.266360    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-j8pjk"]
Sep 11 11:50:48.271926 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:50:48.271900    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0a5db0b_9a22_4973_b161_09d4675e0843.slice/crio-f00425ae1ec12eae297f9d5d0de3f81aa6d2c951a82b774b5f1693e8869a4c12 WatchSource:0}: Error finding container f00425ae1ec12eae297f9d5d0de3f81aa6d2c951a82b774b5f1693e8869a4c12: Status 404 returned error can't find the container with id f00425ae1ec12eae297f9d5d0de3f81aa6d2c951a82b774b5f1693e8869a4c12
Sep 11 11:50:48.275179 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.275154    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:48.482850 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.482819    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:48.687165 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.687121    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:48.810581 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.809237    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 11:50:48.813331 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.813312    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:50:48.816254 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.816219    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 11 11:50:48.816254 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.816220    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 11 11:50:48.816540 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.816510    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:48.816875 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.816859    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 11 11:50:48.817746 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.817727    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 11 11:50:48.817996 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.817982    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 11 11:50:48.818069 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.818013    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 11 11:50:48.818174 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.818158    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-dwwnq\""
Sep 11 11:50:48.818248 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.818232    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 11 11:50:48.818307 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.818231    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 11 11:50:48.818307 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.818235    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 11 11:50:48.842185 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.842132    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 11:50:48.847528 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.847499    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/08e855bb-cda7-4093-99d8-e5ce341100d5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"08e855bb-cda7-4093-99d8-e5ce341100d5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:50:48.847686 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.847573    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/08e855bb-cda7-4093-99d8-e5ce341100d5-web-config\") pod \"alertmanager-main-0\" (UID: \"08e855bb-cda7-4093-99d8-e5ce341100d5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:50:48.847686 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.847601    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/08e855bb-cda7-4093-99d8-e5ce341100d5-config-volume\") pod \"alertmanager-main-0\" (UID: \"08e855bb-cda7-4093-99d8-e5ce341100d5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:50:48.847686 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.847627    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqh6l\" (UniqueName: \"kubernetes.io/projected/08e855bb-cda7-4093-99d8-e5ce341100d5-kube-api-access-hqh6l\") pod \"alertmanager-main-0\" (UID: \"08e855bb-cda7-4093-99d8-e5ce341100d5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:50:48.847858 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.847686    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/08e855bb-cda7-4093-99d8-e5ce341100d5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"08e855bb-cda7-4093-99d8-e5ce341100d5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:50:48.847858 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.847718    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/08e855bb-cda7-4093-99d8-e5ce341100d5-config-out\") pod \"alertmanager-main-0\" (UID: \"08e855bb-cda7-4093-99d8-e5ce341100d5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:50:48.847858 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.847748    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/08e855bb-cda7-4093-99d8-e5ce341100d5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"08e855bb-cda7-4093-99d8-e5ce341100d5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:50:48.847858 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.847788    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/08e855bb-cda7-4093-99d8-e5ce341100d5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"08e855bb-cda7-4093-99d8-e5ce341100d5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:50:48.847858 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.847817    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/08e855bb-cda7-4093-99d8-e5ce341100d5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"08e855bb-cda7-4093-99d8-e5ce341100d5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:50:48.848097 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.847855    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08e855bb-cda7-4093-99d8-e5ce341100d5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"08e855bb-cda7-4093-99d8-e5ce341100d5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:50:48.848097 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.847887    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/08e855bb-cda7-4093-99d8-e5ce341100d5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"08e855bb-cda7-4093-99d8-e5ce341100d5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:50:48.848097 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.847961    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/08e855bb-cda7-4093-99d8-e5ce341100d5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"08e855bb-cda7-4093-99d8-e5ce341100d5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:50:48.848097 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.847999    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/08e855bb-cda7-4093-99d8-e5ce341100d5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"08e855bb-cda7-4093-99d8-e5ce341100d5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:50:48.948745 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.948659    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/08e855bb-cda7-4093-99d8-e5ce341100d5-config-out\") pod \"alertmanager-main-0\" (UID: \"08e855bb-cda7-4093-99d8-e5ce341100d5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:50:48.948745 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.948700    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/08e855bb-cda7-4093-99d8-e5ce341100d5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"08e855bb-cda7-4093-99d8-e5ce341100d5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:50:48.948966 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.948808    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/08e855bb-cda7-4093-99d8-e5ce341100d5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"08e855bb-cda7-4093-99d8-e5ce341100d5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:50:48.948966 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.948826    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/08e855bb-cda7-4093-99d8-e5ce341100d5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"08e855bb-cda7-4093-99d8-e5ce341100d5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:50:48.948966 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.948850    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08e855bb-cda7-4093-99d8-e5ce341100d5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"08e855bb-cda7-4093-99d8-e5ce341100d5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:50:48.948966 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.948870    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/08e855bb-cda7-4093-99d8-e5ce341100d5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"08e855bb-cda7-4093-99d8-e5ce341100d5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:50:48.948966 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.948905    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/08e855bb-cda7-4093-99d8-e5ce341100d5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"08e855bb-cda7-4093-99d8-e5ce341100d5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:50:48.948966 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.948925    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/08e855bb-cda7-4093-99d8-e5ce341100d5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"08e855bb-cda7-4093-99d8-e5ce341100d5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:50:48.948966 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.948947    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/08e855bb-cda7-4093-99d8-e5ce341100d5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"08e855bb-cda7-4093-99d8-e5ce341100d5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:50:48.949309 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.948999    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/08e855bb-cda7-4093-99d8-e5ce341100d5-web-config\") pod \"alertmanager-main-0\" (UID: \"08e855bb-cda7-4093-99d8-e5ce341100d5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:50:48.949309 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.949019    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/08e855bb-cda7-4093-99d8-e5ce341100d5-config-volume\") pod \"alertmanager-main-0\" (UID: \"08e855bb-cda7-4093-99d8-e5ce341100d5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:50:48.949309 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.949032    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqh6l\" (UniqueName: \"kubernetes.io/projected/08e855bb-cda7-4093-99d8-e5ce341100d5-kube-api-access-hqh6l\") pod \"alertmanager-main-0\" (UID: \"08e855bb-cda7-4093-99d8-e5ce341100d5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:50:48.949309 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.949050    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/08e855bb-cda7-4093-99d8-e5ce341100d5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"08e855bb-cda7-4093-99d8-e5ce341100d5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:50:48.949309 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:50:48.949219    2579 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Sep 11 11:50:48.949309 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:50:48.949301    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08e855bb-cda7-4093-99d8-e5ce341100d5-secret-alertmanager-main-tls podName:08e855bb-cda7-4093-99d8-e5ce341100d5 nodeName:}" failed. No retries permitted until 2026-09-11 11:50:49.449271039 +0000 UTC m=+153.346712779 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/08e855bb-cda7-4093-99d8-e5ce341100d5-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "08e855bb-cda7-4093-99d8-e5ce341100d5") : secret "alertmanager-main-tls" not found
Sep 11 11:50:48.949630 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.949363    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/08e855bb-cda7-4093-99d8-e5ce341100d5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"08e855bb-cda7-4093-99d8-e5ce341100d5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:50:48.950105 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.950074    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/08e855bb-cda7-4093-99d8-e5ce341100d5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"08e855bb-cda7-4093-99d8-e5ce341100d5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:50:48.951281 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.951251    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08e855bb-cda7-4093-99d8-e5ce341100d5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"08e855bb-cda7-4093-99d8-e5ce341100d5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:50:48.952718 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.952667    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/08e855bb-cda7-4093-99d8-e5ce341100d5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"08e855bb-cda7-4093-99d8-e5ce341100d5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:50:48.953140 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.952830    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/08e855bb-cda7-4093-99d8-e5ce341100d5-config-out\") pod \"alertmanager-main-0\" (UID: \"08e855bb-cda7-4093-99d8-e5ce341100d5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:50:48.953140 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.953102    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/08e855bb-cda7-4093-99d8-e5ce341100d5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"08e855bb-cda7-4093-99d8-e5ce341100d5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:50:48.953291 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.953202    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/08e855bb-cda7-4093-99d8-e5ce341100d5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"08e855bb-cda7-4093-99d8-e5ce341100d5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:50:48.953731 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.953701    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/08e855bb-cda7-4093-99d8-e5ce341100d5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"08e855bb-cda7-4093-99d8-e5ce341100d5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:50:48.953831 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.953730    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/08e855bb-cda7-4093-99d8-e5ce341100d5-web-config\") pod \"alertmanager-main-0\" (UID: \"08e855bb-cda7-4093-99d8-e5ce341100d5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:50:48.953885 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.953873    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/08e855bb-cda7-4093-99d8-e5ce341100d5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"08e855bb-cda7-4093-99d8-e5ce341100d5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:50:48.954303 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.954271    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/08e855bb-cda7-4093-99d8-e5ce341100d5-config-volume\") pod \"alertmanager-main-0\" (UID: \"08e855bb-cda7-4093-99d8-e5ce341100d5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:50:48.971971 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:48.971884    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqh6l\" (UniqueName: \"kubernetes.io/projected/08e855bb-cda7-4093-99d8-e5ce341100d5-kube-api-access-hqh6l\") pod \"alertmanager-main-0\" (UID: \"08e855bb-cda7-4093-99d8-e5ce341100d5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:50:49.130661 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:49.130612    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-p97jw" event={"ID":"0af7a15f-31e5-4f13-85f5-0879569febcc","Type":"ContainerStarted","Data":"74ff046d89f4375fe539a74c59a9f5f9d8ee2b73e226bca41379e8b7e4cef4de"}
Sep 11 11:50:49.133205 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:49.133159    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-j8pjk" event={"ID":"a0a5db0b-9a22-4973-b161-09d4675e0843","Type":"ContainerStarted","Data":"f963630ab06d617a32716cd7ecd6d01de95265e02c941a6d3c55c7e8fa6e7f76"}
Sep 11 11:50:49.133372 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:49.133227    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-j8pjk" event={"ID":"a0a5db0b-9a22-4973-b161-09d4675e0843","Type":"ContainerStarted","Data":"223bf2c03606fcc1d8a6477d74ba19fe3367a8dd9992bf316a82c5460fb762c0"}
Sep 11 11:50:49.133372 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:49.133261    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-j8pjk" event={"ID":"a0a5db0b-9a22-4973-b161-09d4675e0843","Type":"ContainerStarted","Data":"f00425ae1ec12eae297f9d5d0de3f81aa6d2c951a82b774b5f1693e8869a4c12"}
Sep 11 11:50:49.296277 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:49.296216    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:49.453854 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:49.453783    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/08e855bb-cda7-4093-99d8-e5ce341100d5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"08e855bb-cda7-4093-99d8-e5ce341100d5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:50:49.456154 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:49.456121    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/08e855bb-cda7-4093-99d8-e5ce341100d5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"08e855bb-cda7-4093-99d8-e5ce341100d5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:50:49.723369 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:49.723273    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:50:50.012973 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:50.012887    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 11:50:50.015812 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:50:50.015771    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08e855bb_cda7_4093_99d8_e5ce341100d5.slice/crio-2d9d2665ca89ec1c7bae2ad94a24a178812d7c721ab70ffb0cbc88f8374472b2 WatchSource:0}: Error finding container 2d9d2665ca89ec1c7bae2ad94a24a178812d7c721ab70ffb0cbc88f8374472b2: Status 404 returned error can't find the container with id 2d9d2665ca89ec1c7bae2ad94a24a178812d7c721ab70ffb0cbc88f8374472b2
Sep 11 11:50:50.138367 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:50.138328    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"08e855bb-cda7-4093-99d8-e5ce341100d5","Type":"ContainerStarted","Data":"2d9d2665ca89ec1c7bae2ad94a24a178812d7c721ab70ffb0cbc88f8374472b2"}
Sep 11 11:50:50.139900 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:50.139871    2579 generic.go:356] "Generic (PLEG): container finished" podID="0af7a15f-31e5-4f13-85f5-0879569febcc" containerID="805c86aaf31a770869c32fcac08255d24c29872f5d09f59c77c2ad082d62d14a" exitCode=0
Sep 11 11:50:50.140024 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:50.139931    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-p97jw" event={"ID":"0af7a15f-31e5-4f13-85f5-0879569febcc","Type":"ContainerDied","Data":"805c86aaf31a770869c32fcac08255d24c29872f5d09f59c77c2ad082d62d14a"}
Sep 11 11:50:50.142148 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:50.142126    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-j8pjk" event={"ID":"a0a5db0b-9a22-4973-b161-09d4675e0843","Type":"ContainerStarted","Data":"640d480d1eaa2ffe53930caeba6104d5f75c6da540c7dfb3cb883c9e4c28c4b8"}
Sep 11 11:50:50.189575 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:50.189527    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-j8pjk" podStartSLOduration=2.122022546 podStartE2EDuration="3.189507437s" totalImagesPullingTime="1.067484891s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:50:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:50:50.187792482 +0000 UTC m=+154.085234233" watchObservedRunningTime="2026-09-11 11:50:50.189507437 +0000 UTC m=+154.086949207"
Sep 11 11:50:50.658893 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:50.658855    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:50.759603 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:50.759555    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:50.784213 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:50.784177    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-56bbffc5d9-7z4hv"]
Sep 11 11:50:50.787024 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:50.787001    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-56bbffc5d9-7z4hv"
Sep 11 11:50:50.790932 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:50.790862    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 11 11:50:50.790932 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:50.790912    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 11 11:50:50.791629 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:50.791606    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 11 11:50:50.791976 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:50.791918    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-8hww5\""
Sep 11 11:50:50.792073 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:50.791975    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-81k5tlr3cb8qv\""
Sep 11 11:50:50.792073 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:50.792004    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 11 11:50:50.792267 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:50.792250    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 11 11:50:50.801746 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:50.801722    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-56bbffc5d9-7z4hv"]
Sep 11 11:50:50.859729 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:50.859681    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:50.866558 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:50.866528    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3da41047-d209-4ff5-afac-ad41cf4333c7-secret-grpc-tls\") pod \"thanos-querier-56bbffc5d9-7z4hv\" (UID: \"3da41047-d209-4ff5-afac-ad41cf4333c7\") " pod="openshift-monitoring/thanos-querier-56bbffc5d9-7z4hv"
Sep 11 11:50:50.866706 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:50.866580    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3da41047-d209-4ff5-afac-ad41cf4333c7-metrics-client-ca\") pod \"thanos-querier-56bbffc5d9-7z4hv\" (UID: \"3da41047-d209-4ff5-afac-ad41cf4333c7\") " pod="openshift-monitoring/thanos-querier-56bbffc5d9-7z4hv"
Sep 11 11:50:50.866706 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:50.866665    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3da41047-d209-4ff5-afac-ad41cf4333c7-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-56bbffc5d9-7z4hv\" (UID: \"3da41047-d209-4ff5-afac-ad41cf4333c7\") " pod="openshift-monitoring/thanos-querier-56bbffc5d9-7z4hv"
Sep 11 11:50:50.866805 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:50.866714    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3da41047-d209-4ff5-afac-ad41cf4333c7-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-56bbffc5d9-7z4hv\" (UID: \"3da41047-d209-4ff5-afac-ad41cf4333c7\") " pod="openshift-monitoring/thanos-querier-56bbffc5d9-7z4hv"
Sep 11 11:50:50.866805 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:50.866780    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3da41047-d209-4ff5-afac-ad41cf4333c7-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-56bbffc5d9-7z4hv\" (UID: \"3da41047-d209-4ff5-afac-ad41cf4333c7\") " pod="openshift-monitoring/thanos-querier-56bbffc5d9-7z4hv"
Sep 11 11:50:50.866917 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:50.866877    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3da41047-d209-4ff5-afac-ad41cf4333c7-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-56bbffc5d9-7z4hv\" (UID: \"3da41047-d209-4ff5-afac-ad41cf4333c7\") " pod="openshift-monitoring/thanos-querier-56bbffc5d9-7z4hv"
Sep 11 11:50:50.866984 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:50.866927    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3da41047-d209-4ff5-afac-ad41cf4333c7-secret-thanos-querier-tls\") pod \"thanos-querier-56bbffc5d9-7z4hv\" (UID: \"3da41047-d209-4ff5-afac-ad41cf4333c7\") " pod="openshift-monitoring/thanos-querier-56bbffc5d9-7z4hv"
Sep 11 11:50:50.866984 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:50.866960    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sk44w\" (UniqueName: \"kubernetes.io/projected/3da41047-d209-4ff5-afac-ad41cf4333c7-kube-api-access-sk44w\") pod \"thanos-querier-56bbffc5d9-7z4hv\" (UID: \"3da41047-d209-4ff5-afac-ad41cf4333c7\") " pod="openshift-monitoring/thanos-querier-56bbffc5d9-7z4hv"
Sep 11 11:50:50.968203 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:50.968104    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3da41047-d209-4ff5-afac-ad41cf4333c7-secret-grpc-tls\") pod \"thanos-querier-56bbffc5d9-7z4hv\" (UID: \"3da41047-d209-4ff5-afac-ad41cf4333c7\") " pod="openshift-monitoring/thanos-querier-56bbffc5d9-7z4hv"
Sep 11 11:50:50.968203 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:50.968165    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3da41047-d209-4ff5-afac-ad41cf4333c7-metrics-client-ca\") pod \"thanos-querier-56bbffc5d9-7z4hv\" (UID: \"3da41047-d209-4ff5-afac-ad41cf4333c7\") " pod="openshift-monitoring/thanos-querier-56bbffc5d9-7z4hv"
Sep 11 11:50:50.968203 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:50.968199    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3da41047-d209-4ff5-afac-ad41cf4333c7-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-56bbffc5d9-7z4hv\" (UID: \"3da41047-d209-4ff5-afac-ad41cf4333c7\") " pod="openshift-monitoring/thanos-querier-56bbffc5d9-7z4hv"
Sep 11 11:50:50.968495 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:50.968229    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3da41047-d209-4ff5-afac-ad41cf4333c7-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-56bbffc5d9-7z4hv\" (UID: \"3da41047-d209-4ff5-afac-ad41cf4333c7\") " pod="openshift-monitoring/thanos-querier-56bbffc5d9-7z4hv"
Sep 11 11:50:50.968495 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:50.968245    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3da41047-d209-4ff5-afac-ad41cf4333c7-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-56bbffc5d9-7z4hv\" (UID: \"3da41047-d209-4ff5-afac-ad41cf4333c7\") " pod="openshift-monitoring/thanos-querier-56bbffc5d9-7z4hv"
Sep 11 11:50:50.968495 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:50.968277    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3da41047-d209-4ff5-afac-ad41cf4333c7-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-56bbffc5d9-7z4hv\" (UID: \"3da41047-d209-4ff5-afac-ad41cf4333c7\") " pod="openshift-monitoring/thanos-querier-56bbffc5d9-7z4hv"
Sep 11 11:50:50.968495 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:50.968319    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3da41047-d209-4ff5-afac-ad41cf4333c7-secret-thanos-querier-tls\") pod \"thanos-querier-56bbffc5d9-7z4hv\" (UID: \"3da41047-d209-4ff5-afac-ad41cf4333c7\") " pod="openshift-monitoring/thanos-querier-56bbffc5d9-7z4hv"
Sep 11 11:50:50.968495 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:50.968345    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sk44w\" (UniqueName: \"kubernetes.io/projected/3da41047-d209-4ff5-afac-ad41cf4333c7-kube-api-access-sk44w\") pod \"thanos-querier-56bbffc5d9-7z4hv\" (UID: \"3da41047-d209-4ff5-afac-ad41cf4333c7\") " pod="openshift-monitoring/thanos-querier-56bbffc5d9-7z4hv"
Sep 11 11:50:50.969425 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:50.969378    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3da41047-d209-4ff5-afac-ad41cf4333c7-metrics-client-ca\") pod \"thanos-querier-56bbffc5d9-7z4hv\" (UID: \"3da41047-d209-4ff5-afac-ad41cf4333c7\") " pod="openshift-monitoring/thanos-querier-56bbffc5d9-7z4hv"
Sep 11 11:50:50.971236 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:50.971204    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3da41047-d209-4ff5-afac-ad41cf4333c7-secret-thanos-querier-tls\") pod \"thanos-querier-56bbffc5d9-7z4hv\" (UID: \"3da41047-d209-4ff5-afac-ad41cf4333c7\") " pod="openshift-monitoring/thanos-querier-56bbffc5d9-7z4hv"
Sep 11 11:50:50.971338 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:50.971280    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3da41047-d209-4ff5-afac-ad41cf4333c7-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-56bbffc5d9-7z4hv\" (UID: \"3da41047-d209-4ff5-afac-ad41cf4333c7\") " pod="openshift-monitoring/thanos-querier-56bbffc5d9-7z4hv"
Sep 11 11:50:50.971677 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:50.971652    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3da41047-d209-4ff5-afac-ad41cf4333c7-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-56bbffc5d9-7z4hv\" (UID: \"3da41047-d209-4ff5-afac-ad41cf4333c7\") " pod="openshift-monitoring/thanos-querier-56bbffc5d9-7z4hv"
Sep 11 11:50:50.971828 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:50.971806    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3da41047-d209-4ff5-afac-ad41cf4333c7-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-56bbffc5d9-7z4hv\" (UID: \"3da41047-d209-4ff5-afac-ad41cf4333c7\") " pod="openshift-monitoring/thanos-querier-56bbffc5d9-7z4hv"
Sep 11 11:50:50.971889 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:50.971813    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3da41047-d209-4ff5-afac-ad41cf4333c7-secret-grpc-tls\") pod \"thanos-querier-56bbffc5d9-7z4hv\" (UID: \"3da41047-d209-4ff5-afac-ad41cf4333c7\") " pod="openshift-monitoring/thanos-querier-56bbffc5d9-7z4hv"
Sep 11 11:50:50.971889 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:50.971857    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3da41047-d209-4ff5-afac-ad41cf4333c7-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-56bbffc5d9-7z4hv\" (UID: \"3da41047-d209-4ff5-afac-ad41cf4333c7\") " pod="openshift-monitoring/thanos-querier-56bbffc5d9-7z4hv"
Sep 11 11:50:50.978076 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:50.978055    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sk44w\" (UniqueName: \"kubernetes.io/projected/3da41047-d209-4ff5-afac-ad41cf4333c7-kube-api-access-sk44w\") pod \"thanos-querier-56bbffc5d9-7z4hv\" (UID: \"3da41047-d209-4ff5-afac-ad41cf4333c7\") " pod="openshift-monitoring/thanos-querier-56bbffc5d9-7z4hv"
Sep 11 11:50:51.098051 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:51.098014    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-56bbffc5d9-7z4hv"
Sep 11 11:50:51.137427 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:51.137380    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:51.148988 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:51.148947    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-p97jw" event={"ID":"0af7a15f-31e5-4f13-85f5-0879569febcc","Type":"ContainerStarted","Data":"27a507808c2cadf9512fcbbe8f0ae0d09a4eea3de9444adf34281060a5748334"}
Sep 11 11:50:51.149152 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:51.149016    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-p97jw" event={"ID":"0af7a15f-31e5-4f13-85f5-0879569febcc","Type":"ContainerStarted","Data":"b78cc4a93e0291eb4cf65e917db15cee56d745a168cb8744e78a814b84364177"}
Sep 11 11:50:51.172047 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:51.171991    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-p97jw" podStartSLOduration=3.090401724 podStartE2EDuration="4.17197338s" totalImagesPullingTime="1.011690729s" totalInitContainerRuntime="69.880927ms" isStatefulPod=true podCreationTimestamp="2026-09-11 11:50:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:50:51.170765966 +0000 UTC m=+155.068207713" watchObservedRunningTime="2026-09-11 11:50:51.17197338 +0000 UTC m=+155.069415126"
Sep 11 11:50:51.468872 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:51.468846    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-56bbffc5d9-7z4hv"]
Sep 11 11:50:51.470996 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:50:51.470973    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3da41047_d209_4ff5_afac_ad41cf4333c7.slice/crio-ba1c5ca546898233787a18cc6ab41f7a24fe7352f482c354dac2c32eadff2b9e WatchSource:0}: Error finding container ba1c5ca546898233787a18cc6ab41f7a24fe7352f482c354dac2c32eadff2b9e: Status 404 returned error can't find the container with id ba1c5ca546898233787a18cc6ab41f7a24fe7352f482c354dac2c32eadff2b9e
Sep 11 11:50:52.094083 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:52.094047    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5f945db5-479mq"]
Sep 11 11:50:52.096590 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:52.096568    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5f945db5-479mq"
Sep 11 11:50:52.100800 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:52.100775    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 11 11:50:52.102220 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:52.101983    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-rcxcc\""
Sep 11 11:50:52.102220 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:52.102057    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 11 11:50:52.102220 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:52.102091    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-81p3gh2i0duts\""
Sep 11 11:50:52.102220 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:52.102061    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 11 11:50:52.102220 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:52.102194    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 11 11:50:52.111111 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:52.111085    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5f945db5-479mq"]
Sep 11 11:50:52.155147 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:52.155103    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-56bbffc5d9-7z4hv" event={"ID":"3da41047-d209-4ff5-afac-ad41cf4333c7","Type":"ContainerStarted","Data":"ba1c5ca546898233787a18cc6ab41f7a24fe7352f482c354dac2c32eadff2b9e"}
Sep 11 11:50:52.156902 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:52.156871    2579 generic.go:356] "Generic (PLEG): container finished" podID="08e855bb-cda7-4093-99d8-e5ce341100d5" containerID="2e3b5c83eead778c41af81328f75cf64642a2e4f94ec607ec3efd21c041bf125" exitCode=0
Sep 11 11:50:52.157024 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:52.156905    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"08e855bb-cda7-4093-99d8-e5ce341100d5","Type":"ContainerDied","Data":"2e3b5c83eead778c41af81328f75cf64642a2e4f94ec607ec3efd21c041bf125"}
Sep 11 11:50:52.178331 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:52.178299    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/cab42ad7-002b-4a76-971d-446580ae9467-secret-metrics-server-tls\") pod \"metrics-server-5f945db5-479mq\" (UID: \"cab42ad7-002b-4a76-971d-446580ae9467\") " pod="openshift-monitoring/metrics-server-5f945db5-479mq"
Sep 11 11:50:52.178500 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:52.178371    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/cab42ad7-002b-4a76-971d-446580ae9467-metrics-server-audit-profiles\") pod \"metrics-server-5f945db5-479mq\" (UID: \"cab42ad7-002b-4a76-971d-446580ae9467\") " pod="openshift-monitoring/metrics-server-5f945db5-479mq"
Sep 11 11:50:52.178500 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:52.178480    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7cns\" (UniqueName: \"kubernetes.io/projected/cab42ad7-002b-4a76-971d-446580ae9467-kube-api-access-n7cns\") pod \"metrics-server-5f945db5-479mq\" (UID: \"cab42ad7-002b-4a76-971d-446580ae9467\") " pod="openshift-monitoring/metrics-server-5f945db5-479mq"
Sep 11 11:50:52.178643 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:52.178589    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/cab42ad7-002b-4a76-971d-446580ae9467-secret-metrics-server-client-certs\") pod \"metrics-server-5f945db5-479mq\" (UID: \"cab42ad7-002b-4a76-971d-446580ae9467\") " pod="openshift-monitoring/metrics-server-5f945db5-479mq"
Sep 11 11:50:52.178943 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:52.178897    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/cab42ad7-002b-4a76-971d-446580ae9467-audit-log\") pod \"metrics-server-5f945db5-479mq\" (UID: \"cab42ad7-002b-4a76-971d-446580ae9467\") " pod="openshift-monitoring/metrics-server-5f945db5-479mq"
Sep 11 11:50:52.179055 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:52.178987    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cab42ad7-002b-4a76-971d-446580ae9467-client-ca-bundle\") pod \"metrics-server-5f945db5-479mq\" (UID: \"cab42ad7-002b-4a76-971d-446580ae9467\") " pod="openshift-monitoring/metrics-server-5f945db5-479mq"
Sep 11 11:50:52.179055 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:52.179012    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cab42ad7-002b-4a76-971d-446580ae9467-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5f945db5-479mq\" (UID: \"cab42ad7-002b-4a76-971d-446580ae9467\") " pod="openshift-monitoring/metrics-server-5f945db5-479mq"
Sep 11 11:50:52.280534 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:52.280493    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7cns\" (UniqueName: \"kubernetes.io/projected/cab42ad7-002b-4a76-971d-446580ae9467-kube-api-access-n7cns\") pod \"metrics-server-5f945db5-479mq\" (UID: \"cab42ad7-002b-4a76-971d-446580ae9467\") " pod="openshift-monitoring/metrics-server-5f945db5-479mq"
Sep 11 11:50:52.280724 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:52.280551    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/cab42ad7-002b-4a76-971d-446580ae9467-secret-metrics-server-client-certs\") pod \"metrics-server-5f945db5-479mq\" (UID: \"cab42ad7-002b-4a76-971d-446580ae9467\") " pod="openshift-monitoring/metrics-server-5f945db5-479mq"
Sep 11 11:50:52.280724 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:52.280593    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/cab42ad7-002b-4a76-971d-446580ae9467-audit-log\") pod \"metrics-server-5f945db5-479mq\" (UID: \"cab42ad7-002b-4a76-971d-446580ae9467\") " pod="openshift-monitoring/metrics-server-5f945db5-479mq"
Sep 11 11:50:52.280724 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:52.280630    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cab42ad7-002b-4a76-971d-446580ae9467-client-ca-bundle\") pod \"metrics-server-5f945db5-479mq\" (UID: \"cab42ad7-002b-4a76-971d-446580ae9467\") " pod="openshift-monitoring/metrics-server-5f945db5-479mq"
Sep 11 11:50:52.280724 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:52.280656    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cab42ad7-002b-4a76-971d-446580ae9467-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5f945db5-479mq\" (UID: \"cab42ad7-002b-4a76-971d-446580ae9467\") " pod="openshift-monitoring/metrics-server-5f945db5-479mq"
Sep 11 11:50:52.280724 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:52.280686    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/cab42ad7-002b-4a76-971d-446580ae9467-secret-metrics-server-tls\") pod \"metrics-server-5f945db5-479mq\" (UID: \"cab42ad7-002b-4a76-971d-446580ae9467\") " pod="openshift-monitoring/metrics-server-5f945db5-479mq"
Sep 11 11:50:52.280724 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:52.280719    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/cab42ad7-002b-4a76-971d-446580ae9467-metrics-server-audit-profiles\") pod \"metrics-server-5f945db5-479mq\" (UID: \"cab42ad7-002b-4a76-971d-446580ae9467\") " pod="openshift-monitoring/metrics-server-5f945db5-479mq"
Sep 11 11:50:52.281325 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:52.281041    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/cab42ad7-002b-4a76-971d-446580ae9467-audit-log\") pod \"metrics-server-5f945db5-479mq\" (UID: \"cab42ad7-002b-4a76-971d-446580ae9467\") " pod="openshift-monitoring/metrics-server-5f945db5-479mq"
Sep 11 11:50:52.281763 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:52.281712    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cab42ad7-002b-4a76-971d-446580ae9467-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5f945db5-479mq\" (UID: \"cab42ad7-002b-4a76-971d-446580ae9467\") " pod="openshift-monitoring/metrics-server-5f945db5-479mq"
Sep 11 11:50:52.281993 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:52.281946    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/cab42ad7-002b-4a76-971d-446580ae9467-metrics-server-audit-profiles\") pod \"metrics-server-5f945db5-479mq\" (UID: \"cab42ad7-002b-4a76-971d-446580ae9467\") " pod="openshift-monitoring/metrics-server-5f945db5-479mq"
Sep 11 11:50:52.283851 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:52.283827    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cab42ad7-002b-4a76-971d-446580ae9467-client-ca-bundle\") pod \"metrics-server-5f945db5-479mq\" (UID: \"cab42ad7-002b-4a76-971d-446580ae9467\") " pod="openshift-monitoring/metrics-server-5f945db5-479mq"
Sep 11 11:50:52.283952 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:52.283858    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/cab42ad7-002b-4a76-971d-446580ae9467-secret-metrics-server-tls\") pod \"metrics-server-5f945db5-479mq\" (UID: \"cab42ad7-002b-4a76-971d-446580ae9467\") " pod="openshift-monitoring/metrics-server-5f945db5-479mq"
Sep 11 11:50:52.285795 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:52.285777    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/cab42ad7-002b-4a76-971d-446580ae9467-secret-metrics-server-client-certs\") pod \"metrics-server-5f945db5-479mq\" (UID: \"cab42ad7-002b-4a76-971d-446580ae9467\") " pod="openshift-monitoring/metrics-server-5f945db5-479mq"
Sep 11 11:50:52.289636 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:52.289611    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7cns\" (UniqueName: \"kubernetes.io/projected/cab42ad7-002b-4a76-971d-446580ae9467-kube-api-access-n7cns\") pod \"metrics-server-5f945db5-479mq\" (UID: \"cab42ad7-002b-4a76-971d-446580ae9467\") " pod="openshift-monitoring/metrics-server-5f945db5-479mq"
Sep 11 11:50:52.392681 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:52.392595    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:52.408347 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:52.408312    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5f945db5-479mq"
Sep 11 11:50:52.689068 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:52.688983    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5f945db5-479mq"]
Sep 11 11:50:53.225070 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:50:53.225025    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcab42ad7_002b_4a76_971d_446580ae9467.slice/crio-df7a84d37447539992e009f8cdaf9b8f2f51124de9f0bbf416a76e24c81d08cb WatchSource:0}: Error finding container df7a84d37447539992e009f8cdaf9b8f2f51124de9f0bbf416a76e24c81d08cb: Status 404 returned error can't find the container with id df7a84d37447539992e009f8cdaf9b8f2f51124de9f0bbf416a76e24c81d08cb
Sep 11 11:50:53.704859 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:53.704817    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:53.802347 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:53.802318    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:54.003157 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.003119    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 11:50:54.003493 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.003463    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:54.007211 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.007191    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:50:54.011933 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.011913    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 11 11:50:54.016024 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.016000    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 11 11:50:54.016541 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.016525    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 11 11:50:54.016615 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.016524    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 11 11:50:54.018796 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.018605    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-d9v78es4a057a\""
Sep 11 11:50:54.018796 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.018700    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 11 11:50:54.019677 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.019637    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 11 11:50:54.019896 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.019858    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 11 11:50:54.020464 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.020441    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 11 11:50:54.020833 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.020810    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 11 11:50:54.021507 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.021485    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-zxjqj\""
Sep 11 11:50:54.022251 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.022233    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 11 11:50:54.022656 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.022625    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 11 11:50:54.023023 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.023003    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 11 11:50:54.026733 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.026714    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 11 11:50:54.032455 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.032436    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 11 11:50:54.043094 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.043064    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 11:50:54.097187 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.097152    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a803d13c-58b1-4c9f-940d-479f8ac5452a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:50:54.097416 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.097223    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a803d13c-58b1-4c9f-940d-479f8ac5452a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:50:54.097416 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.097268    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a803d13c-58b1-4c9f-940d-479f8ac5452a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:50:54.097416 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.097300    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a803d13c-58b1-4c9f-940d-479f8ac5452a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:50:54.097416 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.097348    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a803d13c-58b1-4c9f-940d-479f8ac5452a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:50:54.097416 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.097370    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a803d13c-58b1-4c9f-940d-479f8ac5452a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:50:54.097416 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.097408    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a803d13c-58b1-4c9f-940d-479f8ac5452a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:50:54.097725 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.097439    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a803d13c-58b1-4c9f-940d-479f8ac5452a-config\") pod \"prometheus-k8s-0\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:50:54.097725 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.097471    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a803d13c-58b1-4c9f-940d-479f8ac5452a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:50:54.097725 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.097491    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a803d13c-58b1-4c9f-940d-479f8ac5452a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:50:54.097725 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.097555    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a803d13c-58b1-4c9f-940d-479f8ac5452a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:50:54.097725 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.097575    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a803d13c-58b1-4c9f-940d-479f8ac5452a-config-out\") pod \"prometheus-k8s-0\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:50:54.097725 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.097599    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a803d13c-58b1-4c9f-940d-479f8ac5452a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:50:54.097725 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.097620    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a803d13c-58b1-4c9f-940d-479f8ac5452a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:50:54.097725 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.097638    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a803d13c-58b1-4c9f-940d-479f8ac5452a-web-config\") pod \"prometheus-k8s-0\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:50:54.097725 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.097662    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a803d13c-58b1-4c9f-940d-479f8ac5452a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:50:54.098148 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.097740    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a803d13c-58b1-4c9f-940d-479f8ac5452a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:50:54.098148 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.097810    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a803d13c-58b1-4c9f-940d-479f8ac5452a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:50:54.098148 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.097845    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a803d13c-58b1-4c9f-940d-479f8ac5452a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:50:54.098148 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.097877    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8ttj\" (UniqueName: \"kubernetes.io/projected/a803d13c-58b1-4c9f-940d-479f8ac5452a-kube-api-access-t8ttj\") pod \"prometheus-k8s-0\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:50:54.167342 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.167298    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5f945db5-479mq" event={"ID":"cab42ad7-002b-4a76-971d-446580ae9467","Type":"ContainerStarted","Data":"df7a84d37447539992e009f8cdaf9b8f2f51124de9f0bbf416a76e24c81d08cb"}
Sep 11 11:50:54.169903 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.169863    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-56bbffc5d9-7z4hv" event={"ID":"3da41047-d209-4ff5-afac-ad41cf4333c7","Type":"ContainerStarted","Data":"4b696ef4e3e682ebc07e27146130b2db69d33f98e3c3b020d24fbab5c9d0763e"}
Sep 11 11:50:54.170042 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.169939    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-56bbffc5d9-7z4hv" event={"ID":"3da41047-d209-4ff5-afac-ad41cf4333c7","Type":"ContainerStarted","Data":"417267c73b0000c4e8ae7142f0fc0b98eb9827b9f297c8e5ffaf4689a6c716af"}
Sep 11 11:50:54.170042 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.169977    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-56bbffc5d9-7z4hv" event={"ID":"3da41047-d209-4ff5-afac-ad41cf4333c7","Type":"ContainerStarted","Data":"81b92a1147ba07cef10e3e46455626f7703368bb9e201743231af73cf88eaeb8"}
Sep 11 11:50:54.173570 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.173545    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"08e855bb-cda7-4093-99d8-e5ce341100d5","Type":"ContainerStarted","Data":"d28f17725c8a9ebab6da5ae03385f36ad80dbb760e031e1b8339b4af074f21d8"}
Sep 11 11:50:54.173707 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.173606    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"08e855bb-cda7-4093-99d8-e5ce341100d5","Type":"ContainerStarted","Data":"0961d8e6ea1ae4eb2f87aca19b3303ec6715d768ab154505723aeb6a056f602f"}
Sep 11 11:50:54.173707 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.173634    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"08e855bb-cda7-4093-99d8-e5ce341100d5","Type":"ContainerStarted","Data":"8473d2b08867643b93d117f990b043003f689a6352127f1d1ea91f155688fdf1"}
Sep 11 11:50:54.173707 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.173675    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"08e855bb-cda7-4093-99d8-e5ce341100d5","Type":"ContainerStarted","Data":"702bb7f967e827610a56e5d7cde7ef83769201a3edbb04e1f52415258e2971f5"}
Sep 11 11:50:54.173826 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.173709    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"08e855bb-cda7-4093-99d8-e5ce341100d5","Type":"ContainerStarted","Data":"87233b0a69b3b38f73bdfcc88440b592592618166c719fccbf0924356cb80e9d"}
Sep 11 11:50:54.198697 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.198659    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a803d13c-58b1-4c9f-940d-479f8ac5452a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:50:54.198859 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.198705    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8ttj\" (UniqueName: \"kubernetes.io/projected/a803d13c-58b1-4c9f-940d-479f8ac5452a-kube-api-access-t8ttj\") pod \"prometheus-k8s-0\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:50:54.198859 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.198727    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a803d13c-58b1-4c9f-940d-479f8ac5452a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:50:54.198859 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.198756    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a803d13c-58b1-4c9f-940d-479f8ac5452a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:50:54.198859 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.198782    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a803d13c-58b1-4c9f-940d-479f8ac5452a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:50:54.198859 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.198799    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a803d13c-58b1-4c9f-940d-479f8ac5452a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:50:54.198859 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.198828    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a803d13c-58b1-4c9f-940d-479f8ac5452a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:50:54.199139 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.199049    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a803d13c-58b1-4c9f-940d-479f8ac5452a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:50:54.199139 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.199091    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a803d13c-58b1-4c9f-940d-479f8ac5452a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:50:54.199139 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.199123    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a803d13c-58b1-4c9f-940d-479f8ac5452a-config\") pod \"prometheus-k8s-0\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:50:54.199139 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.199133    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a803d13c-58b1-4c9f-940d-479f8ac5452a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:50:54.199348 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.199148    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a803d13c-58b1-4c9f-940d-479f8ac5452a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:50:54.199348 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.199189    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a803d13c-58b1-4c9f-940d-479f8ac5452a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:50:54.199348 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.199247    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a803d13c-58b1-4c9f-940d-479f8ac5452a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:50:54.199348 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.199278    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a803d13c-58b1-4c9f-940d-479f8ac5452a-config-out\") pod \"prometheus-k8s-0\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:50:54.199348 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.199304    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a803d13c-58b1-4c9f-940d-479f8ac5452a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:50:54.199348 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.199328    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a803d13c-58b1-4c9f-940d-479f8ac5452a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:50:54.199794 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.199358    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a803d13c-58b1-4c9f-940d-479f8ac5452a-web-config\") pod \"prometheus-k8s-0\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:50:54.199794 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.199430    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a803d13c-58b1-4c9f-940d-479f8ac5452a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:50:54.199794 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.199461    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a803d13c-58b1-4c9f-940d-479f8ac5452a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:50:54.199794 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.199503    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a803d13c-58b1-4c9f-940d-479f8ac5452a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:50:54.199794 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.199678    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a803d13c-58b1-4c9f-940d-479f8ac5452a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:50:54.201162 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.200098    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a803d13c-58b1-4c9f-940d-479f8ac5452a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:50:54.201162 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.200272    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a803d13c-58b1-4c9f-940d-479f8ac5452a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:50:54.201162 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.200422    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a803d13c-58b1-4c9f-940d-479f8ac5452a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:50:54.201162 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.200857    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a803d13c-58b1-4c9f-940d-479f8ac5452a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:50:54.201162 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.200945    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a803d13c-58b1-4c9f-940d-479f8ac5452a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:50:54.203107 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.202273    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a803d13c-58b1-4c9f-940d-479f8ac5452a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:50:54.203107 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.202274    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a803d13c-58b1-4c9f-940d-479f8ac5452a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:50:54.203107 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.202734    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a803d13c-58b1-4c9f-940d-479f8ac5452a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:50:54.203107 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.202741    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a803d13c-58b1-4c9f-940d-479f8ac5452a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:50:54.203107 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.203057    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a803d13c-58b1-4c9f-940d-479f8ac5452a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:50:54.203648 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.203605    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a803d13c-58b1-4c9f-940d-479f8ac5452a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:50:54.203746 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.203708    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a803d13c-58b1-4c9f-940d-479f8ac5452a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:50:54.203956 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.203934    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/a803d13c-58b1-4c9f-940d-479f8ac5452a-config\") pod \"prometheus-k8s-0\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:50:54.204008 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.203988    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a803d13c-58b1-4c9f-940d-479f8ac5452a-web-config\") pod \"prometheus-k8s-0\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:50:54.204111 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.204097    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a803d13c-58b1-4c9f-940d-479f8ac5452a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:50:54.204440 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.204422    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a803d13c-58b1-4c9f-940d-479f8ac5452a-config-out\") pod \"prometheus-k8s-0\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:50:54.204881 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.204865    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a803d13c-58b1-4c9f-940d-479f8ac5452a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:50:54.208374 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.208345    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8ttj\" (UniqueName: \"kubernetes.io/projected/a803d13c-58b1-4c9f-940d-479f8ac5452a-kube-api-access-t8ttj\") pod \"prometheus-k8s-0\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:50:54.317480 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.317442    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:50:54.386607 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:50:54.386564    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-pmhcz" podUID="9246798b-82ff-4c57-97ff-ce4315e9a904"
Sep 11 11:50:54.616841 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:54.616496    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 11:50:54.713600 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:50:54.713521    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda803d13c_58b1_4c9f_940d_479f8ac5452a.slice/crio-22a9b4e0d19ca620b8391ea6ca704e30139132c907f8e4f4cfeedd23972cce9e WatchSource:0}: Error finding container 22a9b4e0d19ca620b8391ea6ca704e30139132c907f8e4f4cfeedd23972cce9e: Status 404 returned error can't find the container with id 22a9b4e0d19ca620b8391ea6ca704e30139132c907f8e4f4cfeedd23972cce9e
Sep 11 11:50:55.178943 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:55.178905    2579 generic.go:356] "Generic (PLEG): container finished" podID="a803d13c-58b1-4c9f-940d-479f8ac5452a" containerID="d57e4d1f6ee29011ce434f53ba3ebd911dcc91d53e2851725f505ffc317cf3bf" exitCode=0
Sep 11 11:50:55.179079 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:55.178983    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a803d13c-58b1-4c9f-940d-479f8ac5452a","Type":"ContainerDied","Data":"d57e4d1f6ee29011ce434f53ba3ebd911dcc91d53e2851725f505ffc317cf3bf"}
Sep 11 11:50:55.179152 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:55.179106    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a803d13c-58b1-4c9f-940d-479f8ac5452a","Type":"ContainerStarted","Data":"22a9b4e0d19ca620b8391ea6ca704e30139132c907f8e4f4cfeedd23972cce9e"}
Sep 11 11:50:55.180760 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:55.180740    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pmhcz"
Sep 11 11:50:55.180879 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:55.180736    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5f945db5-479mq" event={"ID":"cab42ad7-002b-4a76-971d-446580ae9467","Type":"ContainerStarted","Data":"35930aeea822b186ebef551097da8d2c7d7c8eb90cd3fddd7538e3794f83ac35"}
Sep 11 11:50:55.249677 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:55.249631    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5f945db5-479mq" podStartSLOduration=1.7168522099999999 podStartE2EDuration="3.249613301s" totalImagesPullingTime="1.532761091s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:50:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:50:55.249549356 +0000 UTC m=+159.146991100" watchObservedRunningTime="2026-09-11 11:50:55.249613301 +0000 UTC m=+159.147055043"
Sep 11 11:50:56.188977 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:56.188852    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-56bbffc5d9-7z4hv" event={"ID":"3da41047-d209-4ff5-afac-ad41cf4333c7","Type":"ContainerStarted","Data":"ded611b497417347ab2829ccd196b12b6c2026f262d3397793b087d967f230b6"}
Sep 11 11:50:56.188977 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:56.188931    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-56bbffc5d9-7z4hv" event={"ID":"3da41047-d209-4ff5-afac-ad41cf4333c7","Type":"ContainerStarted","Data":"778b1341bd452f466bade471b91952ea289eb382870ff865f7295d91915f6f34"}
Sep 11 11:50:56.188977 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:56.188971    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-56bbffc5d9-7z4hv" event={"ID":"3da41047-d209-4ff5-afac-ad41cf4333c7","Type":"ContainerStarted","Data":"6e683d51cd90c075dca5cd4b2516d7c3158731cbcba7b528572e76fa77cef390"}
Sep 11 11:50:56.193264 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:56.193226    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"08e855bb-cda7-4093-99d8-e5ce341100d5","Type":"ContainerStarted","Data":"c1e16f3e60e6803d11dcd7d09b431cb0a72a776cfe7dcddec963138fa2e7ba25"}
Sep 11 11:50:56.230608 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:56.230558    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-56bbffc5d9-7z4hv" podStartSLOduration=2.978575088 podStartE2EDuration="6.230540955s" totalImagesPullingTime="3.251965867s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:50:50 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:50:56.226643774 +0000 UTC m=+160.124085522" watchObservedRunningTime="2026-09-11 11:50:56.230540955 +0000 UTC m=+160.127982699"
Sep 11 11:50:56.275003 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:56.274928    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.326581678 podStartE2EDuration="8.274905817s" totalImagesPullingTime="3.932982408s" totalInitContainerRuntime="15.341731ms" isStatefulPod=false podCreationTimestamp="2026-09-11 11:50:48 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:50:56.274152925 +0000 UTC m=+160.171594653" watchObservedRunningTime="2026-09-11 11:50:56.274905817 +0000 UTC m=+160.172347563"
Sep 11 11:50:57.197657 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:57.197617    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-56bbffc5d9-7z4hv"
Sep 11 11:50:58.203801 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:58.203622    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a803d13c-58b1-4c9f-940d-479f8ac5452a","Type":"ContainerStarted","Data":"bbe43e550b6589fa72df5ebc2741eb5bca4dc0deab675fdde8513929ca82487e"}
Sep 11 11:50:58.203801 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:58.203715    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a803d13c-58b1-4c9f-940d-479f8ac5452a","Type":"ContainerStarted","Data":"fba058facb69dbe97717da0b29ca26c64b20dfbda734a8ffb0560dbe8a4a1e10"}
Sep 11 11:50:58.212138 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:58.212087    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-56bbffc5d9-7z4hv"
Sep 11 11:50:58.815328 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:58.815293    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:58.945964 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:58.945922    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:59.211350 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:59.211316    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a803d13c-58b1-4c9f-940d-479f8ac5452a","Type":"ContainerStarted","Data":"c0d8094876c9511da3a1b0f62e9434ad4f3a9aabbfd6a26bd2add4456732f821"}
Sep 11 11:50:59.211749 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:59.211442    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a803d13c-58b1-4c9f-940d-479f8ac5452a","Type":"ContainerStarted","Data":"45d543fa62cb57ec92913de00fe0d59052ccb46efc21697c39d2f0c1074dd924"}
Sep 11 11:50:59.211749 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:59.211475    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a803d13c-58b1-4c9f-940d-479f8ac5452a","Type":"ContainerStarted","Data":"df3e6ab708a38bca472d4fcd6fd803b110a88c62c4278e61126827ad7334429f"}
Sep 11 11:50:59.211749 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:59.211499    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a803d13c-58b1-4c9f-940d-479f8ac5452a","Type":"ContainerStarted","Data":"768a2ae9379c66e6a2fa6a4ec59cc3be949f238aa886be4ee86f66b585e2b8cc"}
Sep 11 11:50:59.259431 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:59.259351    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.386546306 podStartE2EDuration="6.259329827s" totalImagesPullingTime="2.850317239s" totalInitContainerRuntime="22.466282ms" isStatefulPod=false podCreationTimestamp="2026-09-11 11:50:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:50:59.257726535 +0000 UTC m=+163.155168313" watchObservedRunningTime="2026-09-11 11:50:59.259329827 +0000 UTC m=+163.156771573"
Sep 11 11:50:59.317732 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:59.317692    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:50:59.349147 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:59.349116    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9246798b-82ff-4c57-97ff-ce4315e9a904-metrics-tls\") pod \"dns-default-pmhcz\" (UID: \"9246798b-82ff-4c57-97ff-ce4315e9a904\") " pod="openshift-dns/dns-default-pmhcz"
Sep 11 11:50:59.351569 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:59.351543    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9246798b-82ff-4c57-97ff-ce4315e9a904-metrics-tls\") pod \"dns-default-pmhcz\" (UID: \"9246798b-82ff-4c57-97ff-ce4315e9a904\") " pod="openshift-dns/dns-default-pmhcz"
Sep 11 11:50:59.384799 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:59.384767    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-m9sw5\""
Sep 11 11:50:59.392542 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:59.392521    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pmhcz"
Sep 11 11:50:59.685072 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:50:59.685039    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-pmhcz"]
Sep 11 11:50:59.688133 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:50:59.688103    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9246798b_82ff_4c57_97ff_ce4315e9a904.slice/crio-ec4dc72a5decb1b5d015441892b51495304ed020d233bebf4695eb358d63f6dd WatchSource:0}: Error finding container ec4dc72a5decb1b5d015441892b51495304ed020d233bebf4695eb358d63f6dd: Status 404 returned error can't find the container with id ec4dc72a5decb1b5d015441892b51495304ed020d233bebf4695eb358d63f6dd
Sep 11 11:51:00.218441 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:00.218379    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pmhcz" event={"ID":"9246798b-82ff-4c57-97ff-ce4315e9a904","Type":"ContainerStarted","Data":"ec4dc72a5decb1b5d015441892b51495304ed020d233bebf4695eb358d63f6dd"}
Sep 11 11:51:00.570271 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:00.570198    2579 scope.go:117] "RemoveContainer" containerID="aa97bdff795abf16011a7e3f8008056ae7520c4ed835aa8ca3a6f903ad99cd38"
Sep 11 11:51:00.883168 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:00.883086    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:00.984098 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:00.984060    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:01.224383 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:01.224308    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-ch96n_1f0f9fa5-cbd1-468b-ae81-82b48c101d37/console-operator/2.log"
Sep 11 11:51:01.224843 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:01.224423    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-ch96n" event={"ID":"1f0f9fa5-cbd1-468b-ae81-82b48c101d37","Type":"ContainerStarted","Data":"6d6824d16eabd2075a5088cc71ca9f879f1426357d0fc43d17a523149e1a1a42"}
Sep 11 11:51:01.246837 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:01.246783    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-696884fb9c-ch96n" podStartSLOduration=53.697326329 podStartE2EDuration="57.246768286s" totalImagesPullingTime="3.549441957s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:50:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:51:01.245369761 +0000 UTC m=+165.142811504" watchObservedRunningTime="2026-09-11 11:51:01.246768286 +0000 UTC m=+165.144210027"
Sep 11 11:51:01.896728 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:01.896691    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:01.973874 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:01.973843    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:02.037889 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:02.037853    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:02.230250 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:02.230155    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pmhcz" event={"ID":"9246798b-82ff-4c57-97ff-ce4315e9a904","Type":"ContainerStarted","Data":"524ea4d316fdcc144cb5d8bc929841073b31b966c513bbdc69fa579e70d6b5c8"}
Sep 11 11:51:02.230624 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:02.230335    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-pmhcz"
Sep 11 11:51:02.230624 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:02.230368    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pmhcz" event={"ID":"9246798b-82ff-4c57-97ff-ce4315e9a904","Type":"ContainerStarted","Data":"37934641bf8dc4b7e1507822fc3653cd0e743e2dabbe7ad130f0bda23eec0fe3"}
Sep 11 11:51:02.252660 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:02.252611    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-pmhcz" podStartSLOduration=129.691432507 podStartE2EDuration="2m11.252594399s" totalImagesPullingTime="1.561161892s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:48:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:51:02.250776445 +0000 UTC m=+166.148218190" watchObservedRunningTime="2026-09-11 11:51:02.252594399 +0000 UTC m=+166.150036143"
Sep 11 11:51:02.256383 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:02.256352    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:02.351132 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:02.351089    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:02.426839 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:02.426808    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:02.561951 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:02.561854    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:02.662902 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:02.662868    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:02.878681 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:02.878643    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:03.377991 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:03.377948    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:03.571663 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:03.571618    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:03.788588 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:03.788504    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:03.862612 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:03.862584    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:03.944622 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:03.944586    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:04.188594 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:04.188558    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:04.253964 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:04.253932    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:06.265794 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:06.265728    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:06.372520 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:06.372486    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:09.276407 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:09.276300    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:09.373133 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:09.373099    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:11.224982 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:11.224949    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-ch96n"
Sep 11 11:51:11.228743 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:11.228721    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-696884fb9c-ch96n"
Sep 11 11:51:12.236858 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:12.236823    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-pmhcz"
Sep 11 11:51:12.408934 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:12.408889    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5f945db5-479mq"
Sep 11 11:51:12.409105 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:12.408956    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5f945db5-479mq"
Sep 11 11:51:13.663561 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:13.663528    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:15.347779 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:15.347742    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:15.410858 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:15.410817    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:17.814458 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:17.814413    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:17.899015 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:17.898979    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:19.303634 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:19.303545    2579 generic.go:356] "Generic (PLEG): container finished" podID="7f0c67d9-ee3f-4f68-9e53-2b6e8b238d5e" containerID="ff35c35e1e1f0447c126f60b431f87667ec37051989f76d7e368885a596cd3f7" exitCode=0
Sep 11 11:51:19.303996 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:19.303626    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-9l6kz" event={"ID":"7f0c67d9-ee3f-4f68-9e53-2b6e8b238d5e","Type":"ContainerDied","Data":"ff35c35e1e1f0447c126f60b431f87667ec37051989f76d7e368885a596cd3f7"}
Sep 11 11:51:19.303996 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:19.303987    2579 scope.go:117] "RemoveContainer" containerID="ff35c35e1e1f0447c126f60b431f87667ec37051989f76d7e368885a596cd3f7"
Sep 11 11:51:20.314284 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:20.314246    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-9l6kz" event={"ID":"7f0c67d9-ee3f-4f68-9e53-2b6e8b238d5e","Type":"ContainerStarted","Data":"6e792a363c49283194c195defc2fed5468898721a0783a67f9ae7b0457c80bf9"}
Sep 11 11:51:21.409889 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:21.409857    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:21.511245 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:21.511204    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:21.891987 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:21.891956    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:21.978978 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:21.978946    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:22.724622 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:22.724588    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:22.818355 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:22.818318    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:29.259040 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:29.258998    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:29.379167 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:29.379136    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:30.011260 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:30.011224    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:30.081191 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:30.081158    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:32.415186 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:32.415159    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5f945db5-479mq"
Sep 11 11:51:32.420158 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:32.420092    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5f945db5-479mq"
Sep 11 11:51:34.049879 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:34.049841    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:34.427582 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:34.427552    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:34.580789 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:34.580754    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:35.388528 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:35.388490    2579 generic.go:356] "Generic (PLEG): container finished" podID="a3d1c5cd-c0aa-446c-91a2-1df64839bf16" containerID="a3b51f513bf6e045c717fc762eb277e4486701d7c340984b73bcc54c2e4897c9" exitCode=0
Sep 11 11:51:35.388927 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:35.388560    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-t4gdk" event={"ID":"a3d1c5cd-c0aa-446c-91a2-1df64839bf16","Type":"ContainerDied","Data":"a3b51f513bf6e045c717fc762eb277e4486701d7c340984b73bcc54c2e4897c9"}
Sep 11 11:51:35.388927 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:35.388903    2579 scope.go:117] "RemoveContainer" containerID="a3b51f513bf6e045c717fc762eb277e4486701d7c340984b73bcc54c2e4897c9"
Sep 11 11:51:36.394884 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:36.394850    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-t4gdk" event={"ID":"a3d1c5cd-c0aa-446c-91a2-1df64839bf16","Type":"ContainerStarted","Data":"a1ad87979c7fda12ecc81680fb02370a1e9979fa9d845dfb5e9588d528c8c286"}
Sep 11 11:51:37.718508 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:37.718472    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:38.036754 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:38.036660    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:52.706322 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:52.706283    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:52.763564 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:52.763525    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:54.318179 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:54.318127    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:51:54.334251 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:54.334222    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:51:54.486542 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:51:54.486514    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:52:02.750328 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:02.750261    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:02.822314 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:02.822278    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:03.880067 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:03.880021    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:03.951835 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:03.951797    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:08.148664 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:08.148632    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 11:52:08.149059 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:08.148926    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="08e855bb-cda7-4093-99d8-e5ce341100d5" containerName="alertmanager" containerID="cri-o://87233b0a69b3b38f73bdfcc88440b592592618166c719fccbf0924356cb80e9d" gracePeriod=120
Sep 11 11:52:08.149059 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:08.148966    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="08e855bb-cda7-4093-99d8-e5ce341100d5" containerName="kube-rbac-proxy-metric" containerID="cri-o://d28f17725c8a9ebab6da5ae03385f36ad80dbb760e031e1b8339b4af074f21d8" gracePeriod=120
Sep 11 11:52:08.149059 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:08.149003    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="08e855bb-cda7-4093-99d8-e5ce341100d5" containerName="prom-label-proxy" containerID="cri-o://c1e16f3e60e6803d11dcd7d09b431cb0a72a776cfe7dcddec963138fa2e7ba25" gracePeriod=120
Sep 11 11:52:08.149059 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:08.149019    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="08e855bb-cda7-4093-99d8-e5ce341100d5" containerName="kube-rbac-proxy-web" containerID="cri-o://8473d2b08867643b93d117f990b043003f689a6352127f1d1ea91f155688fdf1" gracePeriod=120
Sep 11 11:52:08.149260 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:08.149049    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="08e855bb-cda7-4093-99d8-e5ce341100d5" containerName="kube-rbac-proxy" containerID="cri-o://0961d8e6ea1ae4eb2f87aca19b3303ec6715d768ab154505723aeb6a056f602f" gracePeriod=120
Sep 11 11:52:08.149260 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:08.149070    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="08e855bb-cda7-4093-99d8-e5ce341100d5" containerName="config-reloader" containerID="cri-o://702bb7f967e827610a56e5d7cde7ef83769201a3edbb04e1f52415258e2971f5" gracePeriod=120
Sep 11 11:52:08.531239 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:08.531154    2579 generic.go:356] "Generic (PLEG): container finished" podID="08e855bb-cda7-4093-99d8-e5ce341100d5" containerID="c1e16f3e60e6803d11dcd7d09b431cb0a72a776cfe7dcddec963138fa2e7ba25" exitCode=0
Sep 11 11:52:08.531239 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:08.531182    2579 generic.go:356] "Generic (PLEG): container finished" podID="08e855bb-cda7-4093-99d8-e5ce341100d5" containerID="d28f17725c8a9ebab6da5ae03385f36ad80dbb760e031e1b8339b4af074f21d8" exitCode=0
Sep 11 11:52:08.531239 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:08.531189    2579 generic.go:356] "Generic (PLEG): container finished" podID="08e855bb-cda7-4093-99d8-e5ce341100d5" containerID="0961d8e6ea1ae4eb2f87aca19b3303ec6715d768ab154505723aeb6a056f602f" exitCode=0
Sep 11 11:52:08.531239 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:08.531195    2579 generic.go:356] "Generic (PLEG): container finished" podID="08e855bb-cda7-4093-99d8-e5ce341100d5" containerID="702bb7f967e827610a56e5d7cde7ef83769201a3edbb04e1f52415258e2971f5" exitCode=0
Sep 11 11:52:08.531239 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:08.531202    2579 generic.go:356] "Generic (PLEG): container finished" podID="08e855bb-cda7-4093-99d8-e5ce341100d5" containerID="87233b0a69b3b38f73bdfcc88440b592592618166c719fccbf0924356cb80e9d" exitCode=0
Sep 11 11:52:08.531239 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:08.531220    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"08e855bb-cda7-4093-99d8-e5ce341100d5","Type":"ContainerDied","Data":"c1e16f3e60e6803d11dcd7d09b431cb0a72a776cfe7dcddec963138fa2e7ba25"}
Sep 11 11:52:08.531564 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:08.531311    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"08e855bb-cda7-4093-99d8-e5ce341100d5","Type":"ContainerDied","Data":"d28f17725c8a9ebab6da5ae03385f36ad80dbb760e031e1b8339b4af074f21d8"}
Sep 11 11:52:08.531564 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:08.531346    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"08e855bb-cda7-4093-99d8-e5ce341100d5","Type":"ContainerDied","Data":"0961d8e6ea1ae4eb2f87aca19b3303ec6715d768ab154505723aeb6a056f602f"}
Sep 11 11:52:08.531564 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:08.531374    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"08e855bb-cda7-4093-99d8-e5ce341100d5","Type":"ContainerDied","Data":"702bb7f967e827610a56e5d7cde7ef83769201a3edbb04e1f52415258e2971f5"}
Sep 11 11:52:08.531564 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:08.531412    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"08e855bb-cda7-4093-99d8-e5ce341100d5","Type":"ContainerDied","Data":"87233b0a69b3b38f73bdfcc88440b592592618166c719fccbf0924356cb80e9d"}
Sep 11 11:52:09.418509 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:09.418483    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:52:09.538999 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:09.538916    2579 generic.go:356] "Generic (PLEG): container finished" podID="08e855bb-cda7-4093-99d8-e5ce341100d5" containerID="8473d2b08867643b93d117f990b043003f689a6352127f1d1ea91f155688fdf1" exitCode=0
Sep 11 11:52:09.539140 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:09.539008    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"08e855bb-cda7-4093-99d8-e5ce341100d5","Type":"ContainerDied","Data":"8473d2b08867643b93d117f990b043003f689a6352127f1d1ea91f155688fdf1"}
Sep 11 11:52:09.539140 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:09.539052    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:52:09.539140 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:09.539063    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"08e855bb-cda7-4093-99d8-e5ce341100d5","Type":"ContainerDied","Data":"2d9d2665ca89ec1c7bae2ad94a24a178812d7c721ab70ffb0cbc88f8374472b2"}
Sep 11 11:52:09.539140 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:09.539100    2579 scope.go:117] "RemoveContainer" containerID="c1e16f3e60e6803d11dcd7d09b431cb0a72a776cfe7dcddec963138fa2e7ba25"
Sep 11 11:52:09.546874 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:09.546856    2579 scope.go:117] "RemoveContainer" containerID="d28f17725c8a9ebab6da5ae03385f36ad80dbb760e031e1b8339b4af074f21d8"
Sep 11 11:52:09.553921 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:09.553897    2579 scope.go:117] "RemoveContainer" containerID="0961d8e6ea1ae4eb2f87aca19b3303ec6715d768ab154505723aeb6a056f602f"
Sep 11 11:52:09.560696 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:09.560680    2579 scope.go:117] "RemoveContainer" containerID="8473d2b08867643b93d117f990b043003f689a6352127f1d1ea91f155688fdf1"
Sep 11 11:52:09.562734 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:09.562706    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08e855bb-cda7-4093-99d8-e5ce341100d5-alertmanager-trusted-ca-bundle\") pod \"08e855bb-cda7-4093-99d8-e5ce341100d5\" (UID: \"08e855bb-cda7-4093-99d8-e5ce341100d5\") "
Sep 11 11:52:09.562821 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:09.562761    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hqh6l\" (UniqueName: \"kubernetes.io/projected/08e855bb-cda7-4093-99d8-e5ce341100d5-kube-api-access-hqh6l\") pod \"08e855bb-cda7-4093-99d8-e5ce341100d5\" (UID: \"08e855bb-cda7-4093-99d8-e5ce341100d5\") "
Sep 11 11:52:09.562821 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:09.562787    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/08e855bb-cda7-4093-99d8-e5ce341100d5-secret-alertmanager-main-tls\") pod \"08e855bb-cda7-4093-99d8-e5ce341100d5\" (UID: \"08e855bb-cda7-4093-99d8-e5ce341100d5\") "
Sep 11 11:52:09.562821 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:09.562813    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/08e855bb-cda7-4093-99d8-e5ce341100d5-web-config\") pod \"08e855bb-cda7-4093-99d8-e5ce341100d5\" (UID: \"08e855bb-cda7-4093-99d8-e5ce341100d5\") "
Sep 11 11:52:09.562965 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:09.562858    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/08e855bb-cda7-4093-99d8-e5ce341100d5-cluster-tls-config\") pod \"08e855bb-cda7-4093-99d8-e5ce341100d5\" (UID: \"08e855bb-cda7-4093-99d8-e5ce341100d5\") "
Sep 11 11:52:09.562965 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:09.562902    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/08e855bb-cda7-4093-99d8-e5ce341100d5-config-volume\") pod \"08e855bb-cda7-4093-99d8-e5ce341100d5\" (UID: \"08e855bb-cda7-4093-99d8-e5ce341100d5\") "
Sep 11 11:52:09.562965 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:09.562926    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/08e855bb-cda7-4093-99d8-e5ce341100d5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"08e855bb-cda7-4093-99d8-e5ce341100d5\" (UID: \"08e855bb-cda7-4093-99d8-e5ce341100d5\") "
Sep 11 11:52:09.562965 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:09.562951    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/08e855bb-cda7-4093-99d8-e5ce341100d5-metrics-client-ca\") pod \"08e855bb-cda7-4093-99d8-e5ce341100d5\" (UID: \"08e855bb-cda7-4093-99d8-e5ce341100d5\") "
Sep 11 11:52:09.563225 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:09.563010    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/08e855bb-cda7-4093-99d8-e5ce341100d5-tls-assets\") pod \"08e855bb-cda7-4093-99d8-e5ce341100d5\" (UID: \"08e855bb-cda7-4093-99d8-e5ce341100d5\") "
Sep 11 11:52:09.563225 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:09.563047    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/08e855bb-cda7-4093-99d8-e5ce341100d5-secret-alertmanager-kube-rbac-proxy\") pod \"08e855bb-cda7-4093-99d8-e5ce341100d5\" (UID: \"08e855bb-cda7-4093-99d8-e5ce341100d5\") "
Sep 11 11:52:09.563225 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:09.563072    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/08e855bb-cda7-4093-99d8-e5ce341100d5-secret-alertmanager-kube-rbac-proxy-web\") pod \"08e855bb-cda7-4093-99d8-e5ce341100d5\" (UID: \"08e855bb-cda7-4093-99d8-e5ce341100d5\") "
Sep 11 11:52:09.563225 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:09.563109    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/08e855bb-cda7-4093-99d8-e5ce341100d5-config-out\") pod \"08e855bb-cda7-4093-99d8-e5ce341100d5\" (UID: \"08e855bb-cda7-4093-99d8-e5ce341100d5\") "
Sep 11 11:52:09.563225 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:09.563144    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/08e855bb-cda7-4093-99d8-e5ce341100d5-alertmanager-main-db\") pod \"08e855bb-cda7-4093-99d8-e5ce341100d5\" (UID: \"08e855bb-cda7-4093-99d8-e5ce341100d5\") "
Sep 11 11:52:09.563489 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:09.563349    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08e855bb-cda7-4093-99d8-e5ce341100d5-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "08e855bb-cda7-4093-99d8-e5ce341100d5" (UID: "08e855bb-cda7-4093-99d8-e5ce341100d5"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:52:09.563542 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:09.563490    2579 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08e855bb-cda7-4093-99d8-e5ce341100d5-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 11:52:09.563669 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:09.563647    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08e855bb-cda7-4093-99d8-e5ce341100d5-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "08e855bb-cda7-4093-99d8-e5ce341100d5" (UID: "08e855bb-cda7-4093-99d8-e5ce341100d5"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 11:52:09.564429 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:09.564382    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08e855bb-cda7-4093-99d8-e5ce341100d5-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "08e855bb-cda7-4093-99d8-e5ce341100d5" (UID: "08e855bb-cda7-4093-99d8-e5ce341100d5"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:52:09.565941 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:09.565898    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08e855bb-cda7-4093-99d8-e5ce341100d5-kube-api-access-hqh6l" (OuterVolumeSpecName: "kube-api-access-hqh6l") pod "08e855bb-cda7-4093-99d8-e5ce341100d5" (UID: "08e855bb-cda7-4093-99d8-e5ce341100d5"). InnerVolumeSpecName "kube-api-access-hqh6l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 11:52:09.566195 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:09.566140    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/08e855bb-cda7-4093-99d8-e5ce341100d5-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "08e855bb-cda7-4093-99d8-e5ce341100d5" (UID: "08e855bb-cda7-4093-99d8-e5ce341100d5"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:52:09.566788 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:09.566739    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/08e855bb-cda7-4093-99d8-e5ce341100d5-config-volume" (OuterVolumeSpecName: "config-volume") pod "08e855bb-cda7-4093-99d8-e5ce341100d5" (UID: "08e855bb-cda7-4093-99d8-e5ce341100d5"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:52:09.567151 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:09.567121    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/08e855bb-cda7-4093-99d8-e5ce341100d5-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "08e855bb-cda7-4093-99d8-e5ce341100d5" (UID: "08e855bb-cda7-4093-99d8-e5ce341100d5"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:52:09.567317 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:09.567282    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/08e855bb-cda7-4093-99d8-e5ce341100d5-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "08e855bb-cda7-4093-99d8-e5ce341100d5" (UID: "08e855bb-cda7-4093-99d8-e5ce341100d5"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:52:09.567429 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:09.567286    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08e855bb-cda7-4093-99d8-e5ce341100d5-config-out" (OuterVolumeSpecName: "config-out") pod "08e855bb-cda7-4093-99d8-e5ce341100d5" (UID: "08e855bb-cda7-4093-99d8-e5ce341100d5"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 11:52:09.567771 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:09.567742    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/08e855bb-cda7-4093-99d8-e5ce341100d5-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "08e855bb-cda7-4093-99d8-e5ce341100d5" (UID: "08e855bb-cda7-4093-99d8-e5ce341100d5"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:52:09.568191 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:09.568161    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08e855bb-cda7-4093-99d8-e5ce341100d5-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "08e855bb-cda7-4093-99d8-e5ce341100d5" (UID: "08e855bb-cda7-4093-99d8-e5ce341100d5"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 11:52:09.569883 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:09.569861    2579 scope.go:117] "RemoveContainer" containerID="702bb7f967e827610a56e5d7cde7ef83769201a3edbb04e1f52415258e2971f5"
Sep 11 11:52:09.573010 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:09.572983    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/08e855bb-cda7-4093-99d8-e5ce341100d5-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "08e855bb-cda7-4093-99d8-e5ce341100d5" (UID: "08e855bb-cda7-4093-99d8-e5ce341100d5"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:52:09.578115 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:09.578088    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/08e855bb-cda7-4093-99d8-e5ce341100d5-web-config" (OuterVolumeSpecName: "web-config") pod "08e855bb-cda7-4093-99d8-e5ce341100d5" (UID: "08e855bb-cda7-4093-99d8-e5ce341100d5"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:52:09.588039 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:09.588016    2579 scope.go:117] "RemoveContainer" containerID="87233b0a69b3b38f73bdfcc88440b592592618166c719fccbf0924356cb80e9d"
Sep 11 11:52:09.595623 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:09.595606    2579 scope.go:117] "RemoveContainer" containerID="2e3b5c83eead778c41af81328f75cf64642a2e4f94ec607ec3efd21c041bf125"
Sep 11 11:52:09.602869 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:09.602848    2579 scope.go:117] "RemoveContainer" containerID="c1e16f3e60e6803d11dcd7d09b431cb0a72a776cfe7dcddec963138fa2e7ba25"
Sep 11 11:52:09.603159 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:52:09.603139    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1e16f3e60e6803d11dcd7d09b431cb0a72a776cfe7dcddec963138fa2e7ba25\": container with ID starting with c1e16f3e60e6803d11dcd7d09b431cb0a72a776cfe7dcddec963138fa2e7ba25 not found: ID does not exist" containerID="c1e16f3e60e6803d11dcd7d09b431cb0a72a776cfe7dcddec963138fa2e7ba25"
Sep 11 11:52:09.603204 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:09.603168    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1e16f3e60e6803d11dcd7d09b431cb0a72a776cfe7dcddec963138fa2e7ba25"} err="failed to get container status \"c1e16f3e60e6803d11dcd7d09b431cb0a72a776cfe7dcddec963138fa2e7ba25\": rpc error: code = NotFound desc = could not find container \"c1e16f3e60e6803d11dcd7d09b431cb0a72a776cfe7dcddec963138fa2e7ba25\": container with ID starting with c1e16f3e60e6803d11dcd7d09b431cb0a72a776cfe7dcddec963138fa2e7ba25 not found: ID does not exist"
Sep 11 11:52:09.603204 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:09.603200    2579 scope.go:117] "RemoveContainer" containerID="d28f17725c8a9ebab6da5ae03385f36ad80dbb760e031e1b8339b4af074f21d8"
Sep 11 11:52:09.603457 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:52:09.603441    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d28f17725c8a9ebab6da5ae03385f36ad80dbb760e031e1b8339b4af074f21d8\": container with ID starting with d28f17725c8a9ebab6da5ae03385f36ad80dbb760e031e1b8339b4af074f21d8 not found: ID does not exist" containerID="d28f17725c8a9ebab6da5ae03385f36ad80dbb760e031e1b8339b4af074f21d8"
Sep 11 11:52:09.603505 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:09.603462    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d28f17725c8a9ebab6da5ae03385f36ad80dbb760e031e1b8339b4af074f21d8"} err="failed to get container status \"d28f17725c8a9ebab6da5ae03385f36ad80dbb760e031e1b8339b4af074f21d8\": rpc error: code = NotFound desc = could not find container \"d28f17725c8a9ebab6da5ae03385f36ad80dbb760e031e1b8339b4af074f21d8\": container with ID starting with d28f17725c8a9ebab6da5ae03385f36ad80dbb760e031e1b8339b4af074f21d8 not found: ID does not exist"
Sep 11 11:52:09.603505 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:09.603476    2579 scope.go:117] "RemoveContainer" containerID="0961d8e6ea1ae4eb2f87aca19b3303ec6715d768ab154505723aeb6a056f602f"
Sep 11 11:52:09.603706 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:52:09.603689    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0961d8e6ea1ae4eb2f87aca19b3303ec6715d768ab154505723aeb6a056f602f\": container with ID starting with 0961d8e6ea1ae4eb2f87aca19b3303ec6715d768ab154505723aeb6a056f602f not found: ID does not exist" containerID="0961d8e6ea1ae4eb2f87aca19b3303ec6715d768ab154505723aeb6a056f602f"
Sep 11 11:52:09.603747 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:09.603709    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0961d8e6ea1ae4eb2f87aca19b3303ec6715d768ab154505723aeb6a056f602f"} err="failed to get container status \"0961d8e6ea1ae4eb2f87aca19b3303ec6715d768ab154505723aeb6a056f602f\": rpc error: code = NotFound desc = could not find container \"0961d8e6ea1ae4eb2f87aca19b3303ec6715d768ab154505723aeb6a056f602f\": container with ID starting with 0961d8e6ea1ae4eb2f87aca19b3303ec6715d768ab154505723aeb6a056f602f not found: ID does not exist"
Sep 11 11:52:09.603747 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:09.603722    2579 scope.go:117] "RemoveContainer" containerID="8473d2b08867643b93d117f990b043003f689a6352127f1d1ea91f155688fdf1"
Sep 11 11:52:09.604007 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:52:09.603988    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8473d2b08867643b93d117f990b043003f689a6352127f1d1ea91f155688fdf1\": container with ID starting with 8473d2b08867643b93d117f990b043003f689a6352127f1d1ea91f155688fdf1 not found: ID does not exist" containerID="8473d2b08867643b93d117f990b043003f689a6352127f1d1ea91f155688fdf1"
Sep 11 11:52:09.604050 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:09.604016    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8473d2b08867643b93d117f990b043003f689a6352127f1d1ea91f155688fdf1"} err="failed to get container status \"8473d2b08867643b93d117f990b043003f689a6352127f1d1ea91f155688fdf1\": rpc error: code = NotFound desc = could not find container \"8473d2b08867643b93d117f990b043003f689a6352127f1d1ea91f155688fdf1\": container with ID starting with 8473d2b08867643b93d117f990b043003f689a6352127f1d1ea91f155688fdf1 not found: ID does not exist"
Sep 11 11:52:09.604050 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:09.604035    2579 scope.go:117] "RemoveContainer" containerID="702bb7f967e827610a56e5d7cde7ef83769201a3edbb04e1f52415258e2971f5"
Sep 11 11:52:09.604290 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:52:09.604269    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"702bb7f967e827610a56e5d7cde7ef83769201a3edbb04e1f52415258e2971f5\": container with ID starting with 702bb7f967e827610a56e5d7cde7ef83769201a3edbb04e1f52415258e2971f5 not found: ID does not exist" containerID="702bb7f967e827610a56e5d7cde7ef83769201a3edbb04e1f52415258e2971f5"
Sep 11 11:52:09.604375 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:09.604292    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"702bb7f967e827610a56e5d7cde7ef83769201a3edbb04e1f52415258e2971f5"} err="failed to get container status \"702bb7f967e827610a56e5d7cde7ef83769201a3edbb04e1f52415258e2971f5\": rpc error: code = NotFound desc = could not find container \"702bb7f967e827610a56e5d7cde7ef83769201a3edbb04e1f52415258e2971f5\": container with ID starting with 702bb7f967e827610a56e5d7cde7ef83769201a3edbb04e1f52415258e2971f5 not found: ID does not exist"
Sep 11 11:52:09.604375 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:09.604310    2579 scope.go:117] "RemoveContainer" containerID="87233b0a69b3b38f73bdfcc88440b592592618166c719fccbf0924356cb80e9d"
Sep 11 11:52:09.604568 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:52:09.604552    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"87233b0a69b3b38f73bdfcc88440b592592618166c719fccbf0924356cb80e9d\": container with ID starting with 87233b0a69b3b38f73bdfcc88440b592592618166c719fccbf0924356cb80e9d not found: ID does not exist" containerID="87233b0a69b3b38f73bdfcc88440b592592618166c719fccbf0924356cb80e9d"
Sep 11 11:52:09.604609 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:09.604574    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87233b0a69b3b38f73bdfcc88440b592592618166c719fccbf0924356cb80e9d"} err="failed to get container status \"87233b0a69b3b38f73bdfcc88440b592592618166c719fccbf0924356cb80e9d\": rpc error: code = NotFound desc = could not find container \"87233b0a69b3b38f73bdfcc88440b592592618166c719fccbf0924356cb80e9d\": container with ID starting with 87233b0a69b3b38f73bdfcc88440b592592618166c719fccbf0924356cb80e9d not found: ID does not exist"
Sep 11 11:52:09.604609 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:09.604588    2579 scope.go:117] "RemoveContainer" containerID="2e3b5c83eead778c41af81328f75cf64642a2e4f94ec607ec3efd21c041bf125"
Sep 11 11:52:09.604797 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:52:09.604781    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e3b5c83eead778c41af81328f75cf64642a2e4f94ec607ec3efd21c041bf125\": container with ID starting with 2e3b5c83eead778c41af81328f75cf64642a2e4f94ec607ec3efd21c041bf125 not found: ID does not exist" containerID="2e3b5c83eead778c41af81328f75cf64642a2e4f94ec607ec3efd21c041bf125"
Sep 11 11:52:09.604837 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:09.604800    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e3b5c83eead778c41af81328f75cf64642a2e4f94ec607ec3efd21c041bf125"} err="failed to get container status \"2e3b5c83eead778c41af81328f75cf64642a2e4f94ec607ec3efd21c041bf125\": rpc error: code = NotFound desc = could not find container \"2e3b5c83eead778c41af81328f75cf64642a2e4f94ec607ec3efd21c041bf125\": container with ID starting with 2e3b5c83eead778c41af81328f75cf64642a2e4f94ec607ec3efd21c041bf125 not found: ID does not exist"
Sep 11 11:52:09.664719 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:09.664683    2579 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/08e855bb-cda7-4093-99d8-e5ce341100d5-config-volume\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 11:52:09.664719 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:09.664719    2579 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/08e855bb-cda7-4093-99d8-e5ce341100d5-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 11:52:09.664903 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:09.664734    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/08e855bb-cda7-4093-99d8-e5ce341100d5-metrics-client-ca\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 11:52:09.664903 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:09.664743    2579 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/08e855bb-cda7-4093-99d8-e5ce341100d5-tls-assets\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 11:52:09.664903 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:09.664753    2579 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/08e855bb-cda7-4093-99d8-e5ce341100d5-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 11:52:09.664903 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:09.664762    2579 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/08e855bb-cda7-4093-99d8-e5ce341100d5-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 11:52:09.664903 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:09.664771    2579 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/08e855bb-cda7-4093-99d8-e5ce341100d5-config-out\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 11:52:09.664903 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:09.664779    2579 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/08e855bb-cda7-4093-99d8-e5ce341100d5-alertmanager-main-db\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 11:52:09.664903 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:09.664787    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hqh6l\" (UniqueName: \"kubernetes.io/projected/08e855bb-cda7-4093-99d8-e5ce341100d5-kube-api-access-hqh6l\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 11:52:09.664903 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:09.664800    2579 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/08e855bb-cda7-4093-99d8-e5ce341100d5-secret-alertmanager-main-tls\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 11:52:09.664903 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:09.664833    2579 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/08e855bb-cda7-4093-99d8-e5ce341100d5-web-config\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 11:52:09.664903 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:09.664843    2579 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/08e855bb-cda7-4093-99d8-e5ce341100d5-cluster-tls-config\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 11:52:09.864345 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:09.864309    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 11:52:09.868569 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:09.868540    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 11:52:10.574810 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:10.574732    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="08e855bb-cda7-4093-99d8-e5ce341100d5" path="/var/lib/kubelet/pods/08e855bb-cda7-4093-99d8-e5ce341100d5/volumes"
Sep 11 11:52:11.157485 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:11.157448    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:11.214099 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:11.214066    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:12.483898 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:12.483861    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 11:52:12.484303 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:12.484178    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a803d13c-58b1-4c9f-940d-479f8ac5452a" containerName="prometheus" containerID="cri-o://fba058facb69dbe97717da0b29ca26c64b20dfbda734a8ffb0560dbe8a4a1e10" gracePeriod=600
Sep 11 11:52:12.484303 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:12.484215    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a803d13c-58b1-4c9f-940d-479f8ac5452a" containerName="kube-rbac-proxy" containerID="cri-o://45d543fa62cb57ec92913de00fe0d59052ccb46efc21697c39d2f0c1074dd924" gracePeriod=600
Sep 11 11:52:12.484303 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:12.484243    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a803d13c-58b1-4c9f-940d-479f8ac5452a" containerName="kube-rbac-proxy-web" containerID="cri-o://df3e6ab708a38bca472d4fcd6fd803b110a88c62c4278e61126827ad7334429f" gracePeriod=600
Sep 11 11:52:12.484507 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:12.484281    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a803d13c-58b1-4c9f-940d-479f8ac5452a" containerName="config-reloader" containerID="cri-o://bbe43e550b6589fa72df5ebc2741eb5bca4dc0deab675fdde8513929ca82487e" gracePeriod=600
Sep 11 11:52:12.484507 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:12.484245    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a803d13c-58b1-4c9f-940d-479f8ac5452a" containerName="kube-rbac-proxy-thanos" containerID="cri-o://c0d8094876c9511da3a1b0f62e9434ad4f3a9aabbfd6a26bd2add4456732f821" gracePeriod=600
Sep 11 11:52:12.484507 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:12.484245    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a803d13c-58b1-4c9f-940d-479f8ac5452a" containerName="thanos-sidecar" containerID="cri-o://768a2ae9379c66e6a2fa6a4ec59cc3be949f238aa886be4ee86f66b585e2b8cc" gracePeriod=600
Sep 11 11:52:12.729613 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:12.729586    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:52:12.892998 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:12.892963    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a803d13c-58b1-4c9f-940d-479f8ac5452a-configmap-kubelet-serving-ca-bundle\") pod \"a803d13c-58b1-4c9f-940d-479f8ac5452a\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") "
Sep 11 11:52:12.893190 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:12.893010    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a803d13c-58b1-4c9f-940d-479f8ac5452a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"a803d13c-58b1-4c9f-940d-479f8ac5452a\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") "
Sep 11 11:52:12.893190 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:12.893034    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a803d13c-58b1-4c9f-940d-479f8ac5452a-secret-prometheus-k8s-tls\") pod \"a803d13c-58b1-4c9f-940d-479f8ac5452a\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") "
Sep 11 11:52:12.893190 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:12.893078    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a803d13c-58b1-4c9f-940d-479f8ac5452a-config-out\") pod \"a803d13c-58b1-4c9f-940d-479f8ac5452a\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") "
Sep 11 11:52:12.893190 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:12.893093    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a803d13c-58b1-4c9f-940d-479f8ac5452a-prometheus-k8s-db\") pod \"a803d13c-58b1-4c9f-940d-479f8ac5452a\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") "
Sep 11 11:52:12.893190 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:12.893118    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a803d13c-58b1-4c9f-940d-479f8ac5452a-configmap-metrics-client-ca\") pod \"a803d13c-58b1-4c9f-940d-479f8ac5452a\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") "
Sep 11 11:52:12.893190 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:12.893141    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a803d13c-58b1-4c9f-940d-479f8ac5452a-secret-grpc-tls\") pod \"a803d13c-58b1-4c9f-940d-479f8ac5452a\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") "
Sep 11 11:52:12.893190 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:12.893173    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a803d13c-58b1-4c9f-940d-479f8ac5452a-prometheus-k8s-rulefiles-0\") pod \"a803d13c-58b1-4c9f-940d-479f8ac5452a\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") "
Sep 11 11:52:12.893604 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:12.893197    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a803d13c-58b1-4c9f-940d-479f8ac5452a-thanos-prometheus-http-client-file\") pod \"a803d13c-58b1-4c9f-940d-479f8ac5452a\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") "
Sep 11 11:52:12.893604 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:12.893226    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t8ttj\" (UniqueName: \"kubernetes.io/projected/a803d13c-58b1-4c9f-940d-479f8ac5452a-kube-api-access-t8ttj\") pod \"a803d13c-58b1-4c9f-940d-479f8ac5452a\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") "
Sep 11 11:52:12.893604 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:12.893250    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a803d13c-58b1-4c9f-940d-479f8ac5452a-prometheus-k8s-rulefiles-1\") pod \"a803d13c-58b1-4c9f-940d-479f8ac5452a\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") "
Sep 11 11:52:12.893604 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:12.893281    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a803d13c-58b1-4c9f-940d-479f8ac5452a-secret-metrics-client-certs\") pod \"a803d13c-58b1-4c9f-940d-479f8ac5452a\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") "
Sep 11 11:52:12.893604 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:12.893304    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a803d13c-58b1-4c9f-940d-479f8ac5452a-web-config\") pod \"a803d13c-58b1-4c9f-940d-479f8ac5452a\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") "
Sep 11 11:52:12.893604 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:12.893334    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a803d13c-58b1-4c9f-940d-479f8ac5452a-secret-kube-rbac-proxy\") pod \"a803d13c-58b1-4c9f-940d-479f8ac5452a\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") "
Sep 11 11:52:12.893604 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:12.893379    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a803d13c-58b1-4c9f-940d-479f8ac5452a-prometheus-trusted-ca-bundle\") pod \"a803d13c-58b1-4c9f-940d-479f8ac5452a\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") "
Sep 11 11:52:12.893604 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:12.893459    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a803d13c-58b1-4c9f-940d-479f8ac5452a-configmap-serving-certs-ca-bundle\") pod \"a803d13c-58b1-4c9f-940d-479f8ac5452a\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") "
Sep 11 11:52:12.893604 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:12.893488    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a803d13c-58b1-4c9f-940d-479f8ac5452a-config\") pod \"a803d13c-58b1-4c9f-940d-479f8ac5452a\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") "
Sep 11 11:52:12.893604 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:12.893514    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a803d13c-58b1-4c9f-940d-479f8ac5452a-tls-assets\") pod \"a803d13c-58b1-4c9f-940d-479f8ac5452a\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") "
Sep 11 11:52:12.893604 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:12.893539    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a803d13c-58b1-4c9f-940d-479f8ac5452a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"a803d13c-58b1-4c9f-940d-479f8ac5452a\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") "
Sep 11 11:52:12.893604 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:12.893548    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a803d13c-58b1-4c9f-940d-479f8ac5452a-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "a803d13c-58b1-4c9f-940d-479f8ac5452a" (UID: "a803d13c-58b1-4c9f-940d-479f8ac5452a"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:52:12.893604 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:12.893573    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a803d13c-58b1-4c9f-940d-479f8ac5452a-prometheus-k8s-rulefiles-2\") pod \"a803d13c-58b1-4c9f-940d-479f8ac5452a\" (UID: \"a803d13c-58b1-4c9f-940d-479f8ac5452a\") "
Sep 11 11:52:12.894192 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:12.893824    2579 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a803d13c-58b1-4c9f-940d-479f8ac5452a-configmap-metrics-client-ca\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 11:52:12.894192 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:12.893856    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a803d13c-58b1-4c9f-940d-479f8ac5452a-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "a803d13c-58b1-4c9f-940d-479f8ac5452a" (UID: "a803d13c-58b1-4c9f-940d-479f8ac5452a"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:52:12.894192 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:12.894117    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a803d13c-58b1-4c9f-940d-479f8ac5452a-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "a803d13c-58b1-4c9f-940d-479f8ac5452a" (UID: "a803d13c-58b1-4c9f-940d-479f8ac5452a"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:52:12.894362 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:12.894249    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a803d13c-58b1-4c9f-940d-479f8ac5452a-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "a803d13c-58b1-4c9f-940d-479f8ac5452a" (UID: "a803d13c-58b1-4c9f-940d-479f8ac5452a"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:52:12.894483 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:12.894461    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a803d13c-58b1-4c9f-940d-479f8ac5452a-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "a803d13c-58b1-4c9f-940d-479f8ac5452a" (UID: "a803d13c-58b1-4c9f-940d-479f8ac5452a"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:52:12.894923 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:12.894624    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a803d13c-58b1-4c9f-940d-479f8ac5452a-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "a803d13c-58b1-4c9f-940d-479f8ac5452a" (UID: "a803d13c-58b1-4c9f-940d-479f8ac5452a"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 11:52:12.894923 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:12.894659    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a803d13c-58b1-4c9f-940d-479f8ac5452a-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "a803d13c-58b1-4c9f-940d-479f8ac5452a" (UID: "a803d13c-58b1-4c9f-940d-479f8ac5452a"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:52:12.894923 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:12.894667    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a803d13c-58b1-4c9f-940d-479f8ac5452a-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "a803d13c-58b1-4c9f-940d-479f8ac5452a" (UID: "a803d13c-58b1-4c9f-940d-479f8ac5452a"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:52:12.896056 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:12.895901    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a803d13c-58b1-4c9f-940d-479f8ac5452a-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "a803d13c-58b1-4c9f-940d-479f8ac5452a" (UID: "a803d13c-58b1-4c9f-940d-479f8ac5452a"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:52:12.896056 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:12.895949    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a803d13c-58b1-4c9f-940d-479f8ac5452a-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "a803d13c-58b1-4c9f-940d-479f8ac5452a" (UID: "a803d13c-58b1-4c9f-940d-479f8ac5452a"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:52:12.896270 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:12.896240    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a803d13c-58b1-4c9f-940d-479f8ac5452a-config-out" (OuterVolumeSpecName: "config-out") pod "a803d13c-58b1-4c9f-940d-479f8ac5452a" (UID: "a803d13c-58b1-4c9f-940d-479f8ac5452a"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 11:52:12.896780 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:12.896754    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a803d13c-58b1-4c9f-940d-479f8ac5452a-config" (OuterVolumeSpecName: "config") pod "a803d13c-58b1-4c9f-940d-479f8ac5452a" (UID: "a803d13c-58b1-4c9f-940d-479f8ac5452a"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:52:12.897138 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:12.897072    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a803d13c-58b1-4c9f-940d-479f8ac5452a-kube-api-access-t8ttj" (OuterVolumeSpecName: "kube-api-access-t8ttj") pod "a803d13c-58b1-4c9f-940d-479f8ac5452a" (UID: "a803d13c-58b1-4c9f-940d-479f8ac5452a"). InnerVolumeSpecName "kube-api-access-t8ttj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 11:52:12.897138 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:12.897109    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a803d13c-58b1-4c9f-940d-479f8ac5452a-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "a803d13c-58b1-4c9f-940d-479f8ac5452a" (UID: "a803d13c-58b1-4c9f-940d-479f8ac5452a"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:52:12.897357 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:12.897330    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a803d13c-58b1-4c9f-940d-479f8ac5452a-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "a803d13c-58b1-4c9f-940d-479f8ac5452a" (UID: "a803d13c-58b1-4c9f-940d-479f8ac5452a"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:52:12.897472 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:12.897382    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a803d13c-58b1-4c9f-940d-479f8ac5452a-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "a803d13c-58b1-4c9f-940d-479f8ac5452a" (UID: "a803d13c-58b1-4c9f-940d-479f8ac5452a"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:52:12.897890 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:12.897874    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a803d13c-58b1-4c9f-940d-479f8ac5452a-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "a803d13c-58b1-4c9f-940d-479f8ac5452a" (UID: "a803d13c-58b1-4c9f-940d-479f8ac5452a"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 11:52:12.898355 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:12.898333    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a803d13c-58b1-4c9f-940d-479f8ac5452a-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "a803d13c-58b1-4c9f-940d-479f8ac5452a" (UID: "a803d13c-58b1-4c9f-940d-479f8ac5452a"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:52:12.898822 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:12.898803    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a803d13c-58b1-4c9f-940d-479f8ac5452a-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "a803d13c-58b1-4c9f-940d-479f8ac5452a" (UID: "a803d13c-58b1-4c9f-940d-479f8ac5452a"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:52:12.908051 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:12.908028    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a803d13c-58b1-4c9f-940d-479f8ac5452a-web-config" (OuterVolumeSpecName: "web-config") pod "a803d13c-58b1-4c9f-940d-479f8ac5452a" (UID: "a803d13c-58b1-4c9f-940d-479f8ac5452a"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:52:12.994218 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:12.994164    2579 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a803d13c-58b1-4c9f-940d-479f8ac5452a-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 11:52:12.994218 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:12.994211    2579 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a803d13c-58b1-4c9f-940d-479f8ac5452a-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 11:52:12.994218 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:12.994222    2579 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a803d13c-58b1-4c9f-940d-479f8ac5452a-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 11:52:12.994218 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:12.994232    2579 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a803d13c-58b1-4c9f-940d-479f8ac5452a-secret-prometheus-k8s-tls\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 11:52:12.994501 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:12.994241    2579 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a803d13c-58b1-4c9f-940d-479f8ac5452a-config-out\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 11:52:12.994501 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:12.994249    2579 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a803d13c-58b1-4c9f-940d-479f8ac5452a-prometheus-k8s-db\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 11:52:12.994501 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:12.994257    2579 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a803d13c-58b1-4c9f-940d-479f8ac5452a-secret-grpc-tls\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 11:52:12.994501 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:12.994266    2579 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a803d13c-58b1-4c9f-940d-479f8ac5452a-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 11:52:12.994501 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:12.994275    2579 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a803d13c-58b1-4c9f-940d-479f8ac5452a-thanos-prometheus-http-client-file\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 11:52:12.994501 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:12.994285    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t8ttj\" (UniqueName: \"kubernetes.io/projected/a803d13c-58b1-4c9f-940d-479f8ac5452a-kube-api-access-t8ttj\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 11:52:12.994501 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:12.994293    2579 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a803d13c-58b1-4c9f-940d-479f8ac5452a-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 11:52:12.994501 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:12.994303    2579 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a803d13c-58b1-4c9f-940d-479f8ac5452a-secret-metrics-client-certs\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 11:52:12.994501 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:12.994313    2579 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a803d13c-58b1-4c9f-940d-479f8ac5452a-web-config\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 11:52:12.994501 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:12.994321    2579 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a803d13c-58b1-4c9f-940d-479f8ac5452a-secret-kube-rbac-proxy\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 11:52:12.994501 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:12.994330    2579 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a803d13c-58b1-4c9f-940d-479f8ac5452a-prometheus-trusted-ca-bundle\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 11:52:12.994501 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:12.994338    2579 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a803d13c-58b1-4c9f-940d-479f8ac5452a-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 11:52:12.994501 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:12.994346    2579 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/a803d13c-58b1-4c9f-940d-479f8ac5452a-config\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 11:52:12.994501 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:12.994353    2579 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a803d13c-58b1-4c9f-940d-479f8ac5452a-tls-assets\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 11:52:12.994501 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:12.994362    2579 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a803d13c-58b1-4c9f-940d-479f8ac5452a-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 11:52:13.323355 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.323275    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:13.559175 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.559139    2579 generic.go:356] "Generic (PLEG): container finished" podID="a803d13c-58b1-4c9f-940d-479f8ac5452a" containerID="c0d8094876c9511da3a1b0f62e9434ad4f3a9aabbfd6a26bd2add4456732f821" exitCode=0
Sep 11 11:52:13.559175 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.559164    2579 generic.go:356] "Generic (PLEG): container finished" podID="a803d13c-58b1-4c9f-940d-479f8ac5452a" containerID="45d543fa62cb57ec92913de00fe0d59052ccb46efc21697c39d2f0c1074dd924" exitCode=0
Sep 11 11:52:13.559175 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.559179    2579 generic.go:356] "Generic (PLEG): container finished" podID="a803d13c-58b1-4c9f-940d-479f8ac5452a" containerID="df3e6ab708a38bca472d4fcd6fd803b110a88c62c4278e61126827ad7334429f" exitCode=0
Sep 11 11:52:13.559648 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.559188    2579 generic.go:356] "Generic (PLEG): container finished" podID="a803d13c-58b1-4c9f-940d-479f8ac5452a" containerID="768a2ae9379c66e6a2fa6a4ec59cc3be949f238aa886be4ee86f66b585e2b8cc" exitCode=0
Sep 11 11:52:13.559648 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.559199    2579 generic.go:356] "Generic (PLEG): container finished" podID="a803d13c-58b1-4c9f-940d-479f8ac5452a" containerID="bbe43e550b6589fa72df5ebc2741eb5bca4dc0deab675fdde8513929ca82487e" exitCode=0
Sep 11 11:52:13.559648 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.559207    2579 generic.go:356] "Generic (PLEG): container finished" podID="a803d13c-58b1-4c9f-940d-479f8ac5452a" containerID="fba058facb69dbe97717da0b29ca26c64b20dfbda734a8ffb0560dbe8a4a1e10" exitCode=0
Sep 11 11:52:13.559648 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.559215    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:52:13.559648 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.559182    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a803d13c-58b1-4c9f-940d-479f8ac5452a","Type":"ContainerDied","Data":"c0d8094876c9511da3a1b0f62e9434ad4f3a9aabbfd6a26bd2add4456732f821"}
Sep 11 11:52:13.559648 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.559329    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a803d13c-58b1-4c9f-940d-479f8ac5452a","Type":"ContainerDied","Data":"45d543fa62cb57ec92913de00fe0d59052ccb46efc21697c39d2f0c1074dd924"}
Sep 11 11:52:13.559648 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.559359    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a803d13c-58b1-4c9f-940d-479f8ac5452a","Type":"ContainerDied","Data":"df3e6ab708a38bca472d4fcd6fd803b110a88c62c4278e61126827ad7334429f"}
Sep 11 11:52:13.559648 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.559405    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a803d13c-58b1-4c9f-940d-479f8ac5452a","Type":"ContainerDied","Data":"768a2ae9379c66e6a2fa6a4ec59cc3be949f238aa886be4ee86f66b585e2b8cc"}
Sep 11 11:52:13.559648 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.559412    2579 scope.go:117] "RemoveContainer" containerID="c0d8094876c9511da3a1b0f62e9434ad4f3a9aabbfd6a26bd2add4456732f821"
Sep 11 11:52:13.559648 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.559446    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a803d13c-58b1-4c9f-940d-479f8ac5452a","Type":"ContainerDied","Data":"bbe43e550b6589fa72df5ebc2741eb5bca4dc0deab675fdde8513929ca82487e"}
Sep 11 11:52:13.559648 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.559485    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a803d13c-58b1-4c9f-940d-479f8ac5452a","Type":"ContainerDied","Data":"fba058facb69dbe97717da0b29ca26c64b20dfbda734a8ffb0560dbe8a4a1e10"}
Sep 11 11:52:13.559648 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.559523    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a803d13c-58b1-4c9f-940d-479f8ac5452a","Type":"ContainerDied","Data":"22a9b4e0d19ca620b8391ea6ca704e30139132c907f8e4f4cfeedd23972cce9e"}
Sep 11 11:52:13.567912 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.567892    2579 scope.go:117] "RemoveContainer" containerID="45d543fa62cb57ec92913de00fe0d59052ccb46efc21697c39d2f0c1074dd924"
Sep 11 11:52:13.577056 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.577033    2579 scope.go:117] "RemoveContainer" containerID="df3e6ab708a38bca472d4fcd6fd803b110a88c62c4278e61126827ad7334429f"
Sep 11 11:52:13.584577 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.584557    2579 scope.go:117] "RemoveContainer" containerID="768a2ae9379c66e6a2fa6a4ec59cc3be949f238aa886be4ee86f66b585e2b8cc"
Sep 11 11:52:13.591220 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.591203    2579 scope.go:117] "RemoveContainer" containerID="bbe43e550b6589fa72df5ebc2741eb5bca4dc0deab675fdde8513929ca82487e"
Sep 11 11:52:13.598043 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.598022    2579 scope.go:117] "RemoveContainer" containerID="fba058facb69dbe97717da0b29ca26c64b20dfbda734a8ffb0560dbe8a4a1e10"
Sep 11 11:52:13.601974 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.601953    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 11:52:13.605851 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.605826    2579 scope.go:117] "RemoveContainer" containerID="d57e4d1f6ee29011ce434f53ba3ebd911dcc91d53e2851725f505ffc317cf3bf"
Sep 11 11:52:13.609321 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.609299    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 11:52:13.612991 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.612977    2579 scope.go:117] "RemoveContainer" containerID="c0d8094876c9511da3a1b0f62e9434ad4f3a9aabbfd6a26bd2add4456732f821"
Sep 11 11:52:13.613271 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:52:13.613254    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0d8094876c9511da3a1b0f62e9434ad4f3a9aabbfd6a26bd2add4456732f821\": container with ID starting with c0d8094876c9511da3a1b0f62e9434ad4f3a9aabbfd6a26bd2add4456732f821 not found: ID does not exist" containerID="c0d8094876c9511da3a1b0f62e9434ad4f3a9aabbfd6a26bd2add4456732f821"
Sep 11 11:52:13.613311 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.613277    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0d8094876c9511da3a1b0f62e9434ad4f3a9aabbfd6a26bd2add4456732f821"} err="failed to get container status \"c0d8094876c9511da3a1b0f62e9434ad4f3a9aabbfd6a26bd2add4456732f821\": rpc error: code = NotFound desc = could not find container \"c0d8094876c9511da3a1b0f62e9434ad4f3a9aabbfd6a26bd2add4456732f821\": container with ID starting with c0d8094876c9511da3a1b0f62e9434ad4f3a9aabbfd6a26bd2add4456732f821 not found: ID does not exist"
Sep 11 11:52:13.613311 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.613294    2579 scope.go:117] "RemoveContainer" containerID="45d543fa62cb57ec92913de00fe0d59052ccb46efc21697c39d2f0c1074dd924"
Sep 11 11:52:13.613576 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:52:13.613559    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45d543fa62cb57ec92913de00fe0d59052ccb46efc21697c39d2f0c1074dd924\": container with ID starting with 45d543fa62cb57ec92913de00fe0d59052ccb46efc21697c39d2f0c1074dd924 not found: ID does not exist" containerID="45d543fa62cb57ec92913de00fe0d59052ccb46efc21697c39d2f0c1074dd924"
Sep 11 11:52:13.613635 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.613580    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45d543fa62cb57ec92913de00fe0d59052ccb46efc21697c39d2f0c1074dd924"} err="failed to get container status \"45d543fa62cb57ec92913de00fe0d59052ccb46efc21697c39d2f0c1074dd924\": rpc error: code = NotFound desc = could not find container \"45d543fa62cb57ec92913de00fe0d59052ccb46efc21697c39d2f0c1074dd924\": container with ID starting with 45d543fa62cb57ec92913de00fe0d59052ccb46efc21697c39d2f0c1074dd924 not found: ID does not exist"
Sep 11 11:52:13.613635 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.613599    2579 scope.go:117] "RemoveContainer" containerID="df3e6ab708a38bca472d4fcd6fd803b110a88c62c4278e61126827ad7334429f"
Sep 11 11:52:13.613836 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:52:13.613818    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df3e6ab708a38bca472d4fcd6fd803b110a88c62c4278e61126827ad7334429f\": container with ID starting with df3e6ab708a38bca472d4fcd6fd803b110a88c62c4278e61126827ad7334429f not found: ID does not exist" containerID="df3e6ab708a38bca472d4fcd6fd803b110a88c62c4278e61126827ad7334429f"
Sep 11 11:52:13.613876 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.613841    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df3e6ab708a38bca472d4fcd6fd803b110a88c62c4278e61126827ad7334429f"} err="failed to get container status \"df3e6ab708a38bca472d4fcd6fd803b110a88c62c4278e61126827ad7334429f\": rpc error: code = NotFound desc = could not find container \"df3e6ab708a38bca472d4fcd6fd803b110a88c62c4278e61126827ad7334429f\": container with ID starting with df3e6ab708a38bca472d4fcd6fd803b110a88c62c4278e61126827ad7334429f not found: ID does not exist"
Sep 11 11:52:13.613876 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.613862    2579 scope.go:117] "RemoveContainer" containerID="768a2ae9379c66e6a2fa6a4ec59cc3be949f238aa886be4ee86f66b585e2b8cc"
Sep 11 11:52:13.614107 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:52:13.614091    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"768a2ae9379c66e6a2fa6a4ec59cc3be949f238aa886be4ee86f66b585e2b8cc\": container with ID starting with 768a2ae9379c66e6a2fa6a4ec59cc3be949f238aa886be4ee86f66b585e2b8cc not found: ID does not exist" containerID="768a2ae9379c66e6a2fa6a4ec59cc3be949f238aa886be4ee86f66b585e2b8cc"
Sep 11 11:52:13.614157 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.614108    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"768a2ae9379c66e6a2fa6a4ec59cc3be949f238aa886be4ee86f66b585e2b8cc"} err="failed to get container status \"768a2ae9379c66e6a2fa6a4ec59cc3be949f238aa886be4ee86f66b585e2b8cc\": rpc error: code = NotFound desc = could not find container \"768a2ae9379c66e6a2fa6a4ec59cc3be949f238aa886be4ee86f66b585e2b8cc\": container with ID starting with 768a2ae9379c66e6a2fa6a4ec59cc3be949f238aa886be4ee86f66b585e2b8cc not found: ID does not exist"
Sep 11 11:52:13.614157 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.614129    2579 scope.go:117] "RemoveContainer" containerID="bbe43e550b6589fa72df5ebc2741eb5bca4dc0deab675fdde8513929ca82487e"
Sep 11 11:52:13.614361 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:52:13.614346    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bbe43e550b6589fa72df5ebc2741eb5bca4dc0deab675fdde8513929ca82487e\": container with ID starting with bbe43e550b6589fa72df5ebc2741eb5bca4dc0deab675fdde8513929ca82487e not found: ID does not exist" containerID="bbe43e550b6589fa72df5ebc2741eb5bca4dc0deab675fdde8513929ca82487e"
Sep 11 11:52:13.614427 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.614363    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbe43e550b6589fa72df5ebc2741eb5bca4dc0deab675fdde8513929ca82487e"} err="failed to get container status \"bbe43e550b6589fa72df5ebc2741eb5bca4dc0deab675fdde8513929ca82487e\": rpc error: code = NotFound desc = could not find container \"bbe43e550b6589fa72df5ebc2741eb5bca4dc0deab675fdde8513929ca82487e\": container with ID starting with bbe43e550b6589fa72df5ebc2741eb5bca4dc0deab675fdde8513929ca82487e not found: ID does not exist"
Sep 11 11:52:13.614427 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.614377    2579 scope.go:117] "RemoveContainer" containerID="fba058facb69dbe97717da0b29ca26c64b20dfbda734a8ffb0560dbe8a4a1e10"
Sep 11 11:52:13.614637 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:52:13.614620    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fba058facb69dbe97717da0b29ca26c64b20dfbda734a8ffb0560dbe8a4a1e10\": container with ID starting with fba058facb69dbe97717da0b29ca26c64b20dfbda734a8ffb0560dbe8a4a1e10 not found: ID does not exist" containerID="fba058facb69dbe97717da0b29ca26c64b20dfbda734a8ffb0560dbe8a4a1e10"
Sep 11 11:52:13.614693 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.614640    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fba058facb69dbe97717da0b29ca26c64b20dfbda734a8ffb0560dbe8a4a1e10"} err="failed to get container status \"fba058facb69dbe97717da0b29ca26c64b20dfbda734a8ffb0560dbe8a4a1e10\": rpc error: code = NotFound desc = could not find container \"fba058facb69dbe97717da0b29ca26c64b20dfbda734a8ffb0560dbe8a4a1e10\": container with ID starting with fba058facb69dbe97717da0b29ca26c64b20dfbda734a8ffb0560dbe8a4a1e10 not found: ID does not exist"
Sep 11 11:52:13.614693 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.614654    2579 scope.go:117] "RemoveContainer" containerID="d57e4d1f6ee29011ce434f53ba3ebd911dcc91d53e2851725f505ffc317cf3bf"
Sep 11 11:52:13.614884 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:52:13.614865    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d57e4d1f6ee29011ce434f53ba3ebd911dcc91d53e2851725f505ffc317cf3bf\": container with ID starting with d57e4d1f6ee29011ce434f53ba3ebd911dcc91d53e2851725f505ffc317cf3bf not found: ID does not exist" containerID="d57e4d1f6ee29011ce434f53ba3ebd911dcc91d53e2851725f505ffc317cf3bf"
Sep 11 11:52:13.614922 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.614889    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d57e4d1f6ee29011ce434f53ba3ebd911dcc91d53e2851725f505ffc317cf3bf"} err="failed to get container status \"d57e4d1f6ee29011ce434f53ba3ebd911dcc91d53e2851725f505ffc317cf3bf\": rpc error: code = NotFound desc = could not find container \"d57e4d1f6ee29011ce434f53ba3ebd911dcc91d53e2851725f505ffc317cf3bf\": container with ID starting with d57e4d1f6ee29011ce434f53ba3ebd911dcc91d53e2851725f505ffc317cf3bf not found: ID does not exist"
Sep 11 11:52:13.614922 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.614903    2579 scope.go:117] "RemoveContainer" containerID="c0d8094876c9511da3a1b0f62e9434ad4f3a9aabbfd6a26bd2add4456732f821"
Sep 11 11:52:13.615137 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.615120    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0d8094876c9511da3a1b0f62e9434ad4f3a9aabbfd6a26bd2add4456732f821"} err="failed to get container status \"c0d8094876c9511da3a1b0f62e9434ad4f3a9aabbfd6a26bd2add4456732f821\": rpc error: code = NotFound desc = could not find container \"c0d8094876c9511da3a1b0f62e9434ad4f3a9aabbfd6a26bd2add4456732f821\": container with ID starting with c0d8094876c9511da3a1b0f62e9434ad4f3a9aabbfd6a26bd2add4456732f821 not found: ID does not exist"
Sep 11 11:52:13.615181 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.615145    2579 scope.go:117] "RemoveContainer" containerID="45d543fa62cb57ec92913de00fe0d59052ccb46efc21697c39d2f0c1074dd924"
Sep 11 11:52:13.615356 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.615335    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45d543fa62cb57ec92913de00fe0d59052ccb46efc21697c39d2f0c1074dd924"} err="failed to get container status \"45d543fa62cb57ec92913de00fe0d59052ccb46efc21697c39d2f0c1074dd924\": rpc error: code = NotFound desc = could not find container \"45d543fa62cb57ec92913de00fe0d59052ccb46efc21697c39d2f0c1074dd924\": container with ID starting with 45d543fa62cb57ec92913de00fe0d59052ccb46efc21697c39d2f0c1074dd924 not found: ID does not exist"
Sep 11 11:52:13.615442 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.615358    2579 scope.go:117] "RemoveContainer" containerID="df3e6ab708a38bca472d4fcd6fd803b110a88c62c4278e61126827ad7334429f"
Sep 11 11:52:13.615655 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.615637    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df3e6ab708a38bca472d4fcd6fd803b110a88c62c4278e61126827ad7334429f"} err="failed to get container status \"df3e6ab708a38bca472d4fcd6fd803b110a88c62c4278e61126827ad7334429f\": rpc error: code = NotFound desc = could not find container \"df3e6ab708a38bca472d4fcd6fd803b110a88c62c4278e61126827ad7334429f\": container with ID starting with df3e6ab708a38bca472d4fcd6fd803b110a88c62c4278e61126827ad7334429f not found: ID does not exist"
Sep 11 11:52:13.615725 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.615657    2579 scope.go:117] "RemoveContainer" containerID="768a2ae9379c66e6a2fa6a4ec59cc3be949f238aa886be4ee86f66b585e2b8cc"
Sep 11 11:52:13.615898 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.615880    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"768a2ae9379c66e6a2fa6a4ec59cc3be949f238aa886be4ee86f66b585e2b8cc"} err="failed to get container status \"768a2ae9379c66e6a2fa6a4ec59cc3be949f238aa886be4ee86f66b585e2b8cc\": rpc error: code = NotFound desc = could not find container \"768a2ae9379c66e6a2fa6a4ec59cc3be949f238aa886be4ee86f66b585e2b8cc\": container with ID starting with 768a2ae9379c66e6a2fa6a4ec59cc3be949f238aa886be4ee86f66b585e2b8cc not found: ID does not exist"
Sep 11 11:52:13.615946 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.615898    2579 scope.go:117] "RemoveContainer" containerID="bbe43e550b6589fa72df5ebc2741eb5bca4dc0deab675fdde8513929ca82487e"
Sep 11 11:52:13.616104 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.616089    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbe43e550b6589fa72df5ebc2741eb5bca4dc0deab675fdde8513929ca82487e"} err="failed to get container status \"bbe43e550b6589fa72df5ebc2741eb5bca4dc0deab675fdde8513929ca82487e\": rpc error: code = NotFound desc = could not find container \"bbe43e550b6589fa72df5ebc2741eb5bca4dc0deab675fdde8513929ca82487e\": container with ID starting with bbe43e550b6589fa72df5ebc2741eb5bca4dc0deab675fdde8513929ca82487e not found: ID does not exist"
Sep 11 11:52:13.616104 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.616104    2579 scope.go:117] "RemoveContainer" containerID="fba058facb69dbe97717da0b29ca26c64b20dfbda734a8ffb0560dbe8a4a1e10"
Sep 11 11:52:13.616306 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.616287    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fba058facb69dbe97717da0b29ca26c64b20dfbda734a8ffb0560dbe8a4a1e10"} err="failed to get container status \"fba058facb69dbe97717da0b29ca26c64b20dfbda734a8ffb0560dbe8a4a1e10\": rpc error: code = NotFound desc = could not find container \"fba058facb69dbe97717da0b29ca26c64b20dfbda734a8ffb0560dbe8a4a1e10\": container with ID starting with fba058facb69dbe97717da0b29ca26c64b20dfbda734a8ffb0560dbe8a4a1e10 not found: ID does not exist"
Sep 11 11:52:13.616373 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.616308    2579 scope.go:117] "RemoveContainer" containerID="d57e4d1f6ee29011ce434f53ba3ebd911dcc91d53e2851725f505ffc317cf3bf"
Sep 11 11:52:13.616577 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.616559    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d57e4d1f6ee29011ce434f53ba3ebd911dcc91d53e2851725f505ffc317cf3bf"} err="failed to get container status \"d57e4d1f6ee29011ce434f53ba3ebd911dcc91d53e2851725f505ffc317cf3bf\": rpc error: code = NotFound desc = could not find container \"d57e4d1f6ee29011ce434f53ba3ebd911dcc91d53e2851725f505ffc317cf3bf\": container with ID starting with d57e4d1f6ee29011ce434f53ba3ebd911dcc91d53e2851725f505ffc317cf3bf not found: ID does not exist"
Sep 11 11:52:13.616622 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.616578    2579 scope.go:117] "RemoveContainer" containerID="c0d8094876c9511da3a1b0f62e9434ad4f3a9aabbfd6a26bd2add4456732f821"
Sep 11 11:52:13.616817 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.616798    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0d8094876c9511da3a1b0f62e9434ad4f3a9aabbfd6a26bd2add4456732f821"} err="failed to get container status \"c0d8094876c9511da3a1b0f62e9434ad4f3a9aabbfd6a26bd2add4456732f821\": rpc error: code = NotFound desc = could not find container \"c0d8094876c9511da3a1b0f62e9434ad4f3a9aabbfd6a26bd2add4456732f821\": container with ID starting with c0d8094876c9511da3a1b0f62e9434ad4f3a9aabbfd6a26bd2add4456732f821 not found: ID does not exist"
Sep 11 11:52:13.616889 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.616819    2579 scope.go:117] "RemoveContainer" containerID="45d543fa62cb57ec92913de00fe0d59052ccb46efc21697c39d2f0c1074dd924"
Sep 11 11:52:13.617046 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.617028    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45d543fa62cb57ec92913de00fe0d59052ccb46efc21697c39d2f0c1074dd924"} err="failed to get container status \"45d543fa62cb57ec92913de00fe0d59052ccb46efc21697c39d2f0c1074dd924\": rpc error: code = NotFound desc = could not find container \"45d543fa62cb57ec92913de00fe0d59052ccb46efc21697c39d2f0c1074dd924\": container with ID starting with 45d543fa62cb57ec92913de00fe0d59052ccb46efc21697c39d2f0c1074dd924 not found: ID does not exist"
Sep 11 11:52:13.617088 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.617047    2579 scope.go:117] "RemoveContainer" containerID="df3e6ab708a38bca472d4fcd6fd803b110a88c62c4278e61126827ad7334429f"
Sep 11 11:52:13.617261 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.617244    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df3e6ab708a38bca472d4fcd6fd803b110a88c62c4278e61126827ad7334429f"} err="failed to get container status \"df3e6ab708a38bca472d4fcd6fd803b110a88c62c4278e61126827ad7334429f\": rpc error: code = NotFound desc = could not find container \"df3e6ab708a38bca472d4fcd6fd803b110a88c62c4278e61126827ad7334429f\": container with ID starting with df3e6ab708a38bca472d4fcd6fd803b110a88c62c4278e61126827ad7334429f not found: ID does not exist"
Sep 11 11:52:13.617300 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.617263    2579 scope.go:117] "RemoveContainer" containerID="768a2ae9379c66e6a2fa6a4ec59cc3be949f238aa886be4ee86f66b585e2b8cc"
Sep 11 11:52:13.617525 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.617507    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"768a2ae9379c66e6a2fa6a4ec59cc3be949f238aa886be4ee86f66b585e2b8cc"} err="failed to get container status \"768a2ae9379c66e6a2fa6a4ec59cc3be949f238aa886be4ee86f66b585e2b8cc\": rpc error: code = NotFound desc = could not find container \"768a2ae9379c66e6a2fa6a4ec59cc3be949f238aa886be4ee86f66b585e2b8cc\": container with ID starting with 768a2ae9379c66e6a2fa6a4ec59cc3be949f238aa886be4ee86f66b585e2b8cc not found: ID does not exist"
Sep 11 11:52:13.617604 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.617527    2579 scope.go:117] "RemoveContainer" containerID="bbe43e550b6589fa72df5ebc2741eb5bca4dc0deab675fdde8513929ca82487e"
Sep 11 11:52:13.617780 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.617762    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbe43e550b6589fa72df5ebc2741eb5bca4dc0deab675fdde8513929ca82487e"} err="failed to get container status \"bbe43e550b6589fa72df5ebc2741eb5bca4dc0deab675fdde8513929ca82487e\": rpc error: code = NotFound desc = could not find container \"bbe43e550b6589fa72df5ebc2741eb5bca4dc0deab675fdde8513929ca82487e\": container with ID starting with bbe43e550b6589fa72df5ebc2741eb5bca4dc0deab675fdde8513929ca82487e not found: ID does not exist"
Sep 11 11:52:13.617816 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.617780    2579 scope.go:117] "RemoveContainer" containerID="fba058facb69dbe97717da0b29ca26c64b20dfbda734a8ffb0560dbe8a4a1e10"
Sep 11 11:52:13.617986 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.617967    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fba058facb69dbe97717da0b29ca26c64b20dfbda734a8ffb0560dbe8a4a1e10"} err="failed to get container status \"fba058facb69dbe97717da0b29ca26c64b20dfbda734a8ffb0560dbe8a4a1e10\": rpc error: code = NotFound desc = could not find container \"fba058facb69dbe97717da0b29ca26c64b20dfbda734a8ffb0560dbe8a4a1e10\": container with ID starting with fba058facb69dbe97717da0b29ca26c64b20dfbda734a8ffb0560dbe8a4a1e10 not found: ID does not exist"
Sep 11 11:52:13.618051 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.617988    2579 scope.go:117] "RemoveContainer" containerID="d57e4d1f6ee29011ce434f53ba3ebd911dcc91d53e2851725f505ffc317cf3bf"
Sep 11 11:52:13.618221 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.618205    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d57e4d1f6ee29011ce434f53ba3ebd911dcc91d53e2851725f505ffc317cf3bf"} err="failed to get container status \"d57e4d1f6ee29011ce434f53ba3ebd911dcc91d53e2851725f505ffc317cf3bf\": rpc error: code = NotFound desc = could not find container \"d57e4d1f6ee29011ce434f53ba3ebd911dcc91d53e2851725f505ffc317cf3bf\": container with ID starting with d57e4d1f6ee29011ce434f53ba3ebd911dcc91d53e2851725f505ffc317cf3bf not found: ID does not exist"
Sep 11 11:52:13.618271 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.618221    2579 scope.go:117] "RemoveContainer" containerID="c0d8094876c9511da3a1b0f62e9434ad4f3a9aabbfd6a26bd2add4456732f821"
Sep 11 11:52:13.618474 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.618455    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0d8094876c9511da3a1b0f62e9434ad4f3a9aabbfd6a26bd2add4456732f821"} err="failed to get container status \"c0d8094876c9511da3a1b0f62e9434ad4f3a9aabbfd6a26bd2add4456732f821\": rpc error: code = NotFound desc = could not find container \"c0d8094876c9511da3a1b0f62e9434ad4f3a9aabbfd6a26bd2add4456732f821\": container with ID starting with c0d8094876c9511da3a1b0f62e9434ad4f3a9aabbfd6a26bd2add4456732f821 not found: ID does not exist"
Sep 11 11:52:13.618540 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.618475    2579 scope.go:117] "RemoveContainer" containerID="45d543fa62cb57ec92913de00fe0d59052ccb46efc21697c39d2f0c1074dd924"
Sep 11 11:52:13.618709 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.618691    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45d543fa62cb57ec92913de00fe0d59052ccb46efc21697c39d2f0c1074dd924"} err="failed to get container status \"45d543fa62cb57ec92913de00fe0d59052ccb46efc21697c39d2f0c1074dd924\": rpc error: code = NotFound desc = could not find container \"45d543fa62cb57ec92913de00fe0d59052ccb46efc21697c39d2f0c1074dd924\": container with ID starting with 45d543fa62cb57ec92913de00fe0d59052ccb46efc21697c39d2f0c1074dd924 not found: ID does not exist"
Sep 11 11:52:13.618774 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.618710    2579 scope.go:117] "RemoveContainer" containerID="df3e6ab708a38bca472d4fcd6fd803b110a88c62c4278e61126827ad7334429f"
Sep 11 11:52:13.618962 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.618944    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df3e6ab708a38bca472d4fcd6fd803b110a88c62c4278e61126827ad7334429f"} err="failed to get container status \"df3e6ab708a38bca472d4fcd6fd803b110a88c62c4278e61126827ad7334429f\": rpc error: code = NotFound desc = could not find container \"df3e6ab708a38bca472d4fcd6fd803b110a88c62c4278e61126827ad7334429f\": container with ID starting with df3e6ab708a38bca472d4fcd6fd803b110a88c62c4278e61126827ad7334429f not found: ID does not exist"
Sep 11 11:52:13.619024 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.618963    2579 scope.go:117] "RemoveContainer" containerID="768a2ae9379c66e6a2fa6a4ec59cc3be949f238aa886be4ee86f66b585e2b8cc"
Sep 11 11:52:13.619196 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.619176    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"768a2ae9379c66e6a2fa6a4ec59cc3be949f238aa886be4ee86f66b585e2b8cc"} err="failed to get container status \"768a2ae9379c66e6a2fa6a4ec59cc3be949f238aa886be4ee86f66b585e2b8cc\": rpc error: code = NotFound desc = could not find container \"768a2ae9379c66e6a2fa6a4ec59cc3be949f238aa886be4ee86f66b585e2b8cc\": container with ID starting with 768a2ae9379c66e6a2fa6a4ec59cc3be949f238aa886be4ee86f66b585e2b8cc not found: ID does not exist"
Sep 11 11:52:13.619238 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.619196    2579 scope.go:117] "RemoveContainer" containerID="bbe43e550b6589fa72df5ebc2741eb5bca4dc0deab675fdde8513929ca82487e"
Sep 11 11:52:13.619421 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.619383    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbe43e550b6589fa72df5ebc2741eb5bca4dc0deab675fdde8513929ca82487e"} err="failed to get container status \"bbe43e550b6589fa72df5ebc2741eb5bca4dc0deab675fdde8513929ca82487e\": rpc error: code = NotFound desc = could not find container \"bbe43e550b6589fa72df5ebc2741eb5bca4dc0deab675fdde8513929ca82487e\": container with ID starting with bbe43e550b6589fa72df5ebc2741eb5bca4dc0deab675fdde8513929ca82487e not found: ID does not exist"
Sep 11 11:52:13.619483 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.619423    2579 scope.go:117] "RemoveContainer" containerID="fba058facb69dbe97717da0b29ca26c64b20dfbda734a8ffb0560dbe8a4a1e10"
Sep 11 11:52:13.619660 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.619643    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fba058facb69dbe97717da0b29ca26c64b20dfbda734a8ffb0560dbe8a4a1e10"} err="failed to get container status \"fba058facb69dbe97717da0b29ca26c64b20dfbda734a8ffb0560dbe8a4a1e10\": rpc error: code = NotFound desc = could not find container \"fba058facb69dbe97717da0b29ca26c64b20dfbda734a8ffb0560dbe8a4a1e10\": container with ID starting with fba058facb69dbe97717da0b29ca26c64b20dfbda734a8ffb0560dbe8a4a1e10 not found: ID does not exist"
Sep 11 11:52:13.619707 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.619660    2579 scope.go:117] "RemoveContainer" containerID="d57e4d1f6ee29011ce434f53ba3ebd911dcc91d53e2851725f505ffc317cf3bf"
Sep 11 11:52:13.619883 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.619865    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d57e4d1f6ee29011ce434f53ba3ebd911dcc91d53e2851725f505ffc317cf3bf"} err="failed to get container status \"d57e4d1f6ee29011ce434f53ba3ebd911dcc91d53e2851725f505ffc317cf3bf\": rpc error: code = NotFound desc = could not find container \"d57e4d1f6ee29011ce434f53ba3ebd911dcc91d53e2851725f505ffc317cf3bf\": container with ID starting with d57e4d1f6ee29011ce434f53ba3ebd911dcc91d53e2851725f505ffc317cf3bf not found: ID does not exist"
Sep 11 11:52:13.619951 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.619885    2579 scope.go:117] "RemoveContainer" containerID="c0d8094876c9511da3a1b0f62e9434ad4f3a9aabbfd6a26bd2add4456732f821"
Sep 11 11:52:13.620129 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.620112    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0d8094876c9511da3a1b0f62e9434ad4f3a9aabbfd6a26bd2add4456732f821"} err="failed to get container status \"c0d8094876c9511da3a1b0f62e9434ad4f3a9aabbfd6a26bd2add4456732f821\": rpc error: code = NotFound desc = could not find container \"c0d8094876c9511da3a1b0f62e9434ad4f3a9aabbfd6a26bd2add4456732f821\": container with ID starting with c0d8094876c9511da3a1b0f62e9434ad4f3a9aabbfd6a26bd2add4456732f821 not found: ID does not exist"
Sep 11 11:52:13.620178 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.620130    2579 scope.go:117] "RemoveContainer" containerID="45d543fa62cb57ec92913de00fe0d59052ccb46efc21697c39d2f0c1074dd924"
Sep 11 11:52:13.620343 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.620326    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45d543fa62cb57ec92913de00fe0d59052ccb46efc21697c39d2f0c1074dd924"} err="failed to get container status \"45d543fa62cb57ec92913de00fe0d59052ccb46efc21697c39d2f0c1074dd924\": rpc error: code = NotFound desc = could not find container \"45d543fa62cb57ec92913de00fe0d59052ccb46efc21697c39d2f0c1074dd924\": container with ID starting with 45d543fa62cb57ec92913de00fe0d59052ccb46efc21697c39d2f0c1074dd924 not found: ID does not exist"
Sep 11 11:52:13.620383 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.620342    2579 scope.go:117] "RemoveContainer" containerID="df3e6ab708a38bca472d4fcd6fd803b110a88c62c4278e61126827ad7334429f"
Sep 11 11:52:13.620567 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.620549    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df3e6ab708a38bca472d4fcd6fd803b110a88c62c4278e61126827ad7334429f"} err="failed to get container status \"df3e6ab708a38bca472d4fcd6fd803b110a88c62c4278e61126827ad7334429f\": rpc error: code = NotFound desc = could not find container \"df3e6ab708a38bca472d4fcd6fd803b110a88c62c4278e61126827ad7334429f\": container with ID starting with df3e6ab708a38bca472d4fcd6fd803b110a88c62c4278e61126827ad7334429f not found: ID does not exist"
Sep 11 11:52:13.620628 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.620569    2579 scope.go:117] "RemoveContainer" containerID="768a2ae9379c66e6a2fa6a4ec59cc3be949f238aa886be4ee86f66b585e2b8cc"
Sep 11 11:52:13.620770 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.620755    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"768a2ae9379c66e6a2fa6a4ec59cc3be949f238aa886be4ee86f66b585e2b8cc"} err="failed to get container status \"768a2ae9379c66e6a2fa6a4ec59cc3be949f238aa886be4ee86f66b585e2b8cc\": rpc error: code = NotFound desc = could not find container \"768a2ae9379c66e6a2fa6a4ec59cc3be949f238aa886be4ee86f66b585e2b8cc\": container with ID starting with 768a2ae9379c66e6a2fa6a4ec59cc3be949f238aa886be4ee86f66b585e2b8cc not found: ID does not exist"
Sep 11 11:52:13.620839 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.620770    2579 scope.go:117] "RemoveContainer" containerID="bbe43e550b6589fa72df5ebc2741eb5bca4dc0deab675fdde8513929ca82487e"
Sep 11 11:52:13.620982 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.620965    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbe43e550b6589fa72df5ebc2741eb5bca4dc0deab675fdde8513929ca82487e"} err="failed to get container status \"bbe43e550b6589fa72df5ebc2741eb5bca4dc0deab675fdde8513929ca82487e\": rpc error: code = NotFound desc = could not find container \"bbe43e550b6589fa72df5ebc2741eb5bca4dc0deab675fdde8513929ca82487e\": container with ID starting with bbe43e550b6589fa72df5ebc2741eb5bca4dc0deab675fdde8513929ca82487e not found: ID does not exist"
Sep 11 11:52:13.621047 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.620983    2579 scope.go:117] "RemoveContainer" containerID="fba058facb69dbe97717da0b29ca26c64b20dfbda734a8ffb0560dbe8a4a1e10"
Sep 11 11:52:13.621229 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.621211    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fba058facb69dbe97717da0b29ca26c64b20dfbda734a8ffb0560dbe8a4a1e10"} err="failed to get container status \"fba058facb69dbe97717da0b29ca26c64b20dfbda734a8ffb0560dbe8a4a1e10\": rpc error: code = NotFound desc = could not find container \"fba058facb69dbe97717da0b29ca26c64b20dfbda734a8ffb0560dbe8a4a1e10\": container with ID starting with fba058facb69dbe97717da0b29ca26c64b20dfbda734a8ffb0560dbe8a4a1e10 not found: ID does not exist"
Sep 11 11:52:13.621276 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.621229    2579 scope.go:117] "RemoveContainer" containerID="d57e4d1f6ee29011ce434f53ba3ebd911dcc91d53e2851725f505ffc317cf3bf"
Sep 11 11:52:13.621515 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.621486    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d57e4d1f6ee29011ce434f53ba3ebd911dcc91d53e2851725f505ffc317cf3bf"} err="failed to get container status \"d57e4d1f6ee29011ce434f53ba3ebd911dcc91d53e2851725f505ffc317cf3bf\": rpc error: code = NotFound desc = could not find container \"d57e4d1f6ee29011ce434f53ba3ebd911dcc91d53e2851725f505ffc317cf3bf\": container with ID starting with d57e4d1f6ee29011ce434f53ba3ebd911dcc91d53e2851725f505ffc317cf3bf not found: ID does not exist"
Sep 11 11:52:13.621515 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.621515    2579 scope.go:117] "RemoveContainer" containerID="c0d8094876c9511da3a1b0f62e9434ad4f3a9aabbfd6a26bd2add4456732f821"
Sep 11 11:52:13.621720 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.621702    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0d8094876c9511da3a1b0f62e9434ad4f3a9aabbfd6a26bd2add4456732f821"} err="failed to get container status \"c0d8094876c9511da3a1b0f62e9434ad4f3a9aabbfd6a26bd2add4456732f821\": rpc error: code = NotFound desc = could not find container \"c0d8094876c9511da3a1b0f62e9434ad4f3a9aabbfd6a26bd2add4456732f821\": container with ID starting with c0d8094876c9511da3a1b0f62e9434ad4f3a9aabbfd6a26bd2add4456732f821 not found: ID does not exist"
Sep 11 11:52:13.621779 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.621722    2579 scope.go:117] "RemoveContainer" containerID="45d543fa62cb57ec92913de00fe0d59052ccb46efc21697c39d2f0c1074dd924"
Sep 11 11:52:13.621926 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.621910    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45d543fa62cb57ec92913de00fe0d59052ccb46efc21697c39d2f0c1074dd924"} err="failed to get container status \"45d543fa62cb57ec92913de00fe0d59052ccb46efc21697c39d2f0c1074dd924\": rpc error: code = NotFound desc = could not find container \"45d543fa62cb57ec92913de00fe0d59052ccb46efc21697c39d2f0c1074dd924\": container with ID starting with 45d543fa62cb57ec92913de00fe0d59052ccb46efc21697c39d2f0c1074dd924 not found: ID does not exist"
Sep 11 11:52:13.621973 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.621927    2579 scope.go:117] "RemoveContainer" containerID="df3e6ab708a38bca472d4fcd6fd803b110a88c62c4278e61126827ad7334429f"
Sep 11 11:52:13.622154 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.622139    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df3e6ab708a38bca472d4fcd6fd803b110a88c62c4278e61126827ad7334429f"} err="failed to get container status \"df3e6ab708a38bca472d4fcd6fd803b110a88c62c4278e61126827ad7334429f\": rpc error: code = NotFound desc = could not find container \"df3e6ab708a38bca472d4fcd6fd803b110a88c62c4278e61126827ad7334429f\": container with ID starting with df3e6ab708a38bca472d4fcd6fd803b110a88c62c4278e61126827ad7334429f not found: ID does not exist"
Sep 11 11:52:13.622211 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.622156    2579 scope.go:117] "RemoveContainer" containerID="768a2ae9379c66e6a2fa6a4ec59cc3be949f238aa886be4ee86f66b585e2b8cc"
Sep 11 11:52:13.622403 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.622374    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"768a2ae9379c66e6a2fa6a4ec59cc3be949f238aa886be4ee86f66b585e2b8cc"} err="failed to get container status \"768a2ae9379c66e6a2fa6a4ec59cc3be949f238aa886be4ee86f66b585e2b8cc\": rpc error: code = NotFound desc = could not find container \"768a2ae9379c66e6a2fa6a4ec59cc3be949f238aa886be4ee86f66b585e2b8cc\": container with ID starting with 768a2ae9379c66e6a2fa6a4ec59cc3be949f238aa886be4ee86f66b585e2b8cc not found: ID does not exist"
Sep 11 11:52:13.622456 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.622413    2579 scope.go:117] "RemoveContainer" containerID="bbe43e550b6589fa72df5ebc2741eb5bca4dc0deab675fdde8513929ca82487e"
Sep 11 11:52:13.622627 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.622609    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbe43e550b6589fa72df5ebc2741eb5bca4dc0deab675fdde8513929ca82487e"} err="failed to get container status \"bbe43e550b6589fa72df5ebc2741eb5bca4dc0deab675fdde8513929ca82487e\": rpc error: code = NotFound desc = could not find container \"bbe43e550b6589fa72df5ebc2741eb5bca4dc0deab675fdde8513929ca82487e\": container with ID starting with bbe43e550b6589fa72df5ebc2741eb5bca4dc0deab675fdde8513929ca82487e not found: ID does not exist"
Sep 11 11:52:13.622687 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.622629    2579 scope.go:117] "RemoveContainer" containerID="fba058facb69dbe97717da0b29ca26c64b20dfbda734a8ffb0560dbe8a4a1e10"
Sep 11 11:52:13.622860 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.622843    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fba058facb69dbe97717da0b29ca26c64b20dfbda734a8ffb0560dbe8a4a1e10"} err="failed to get container status \"fba058facb69dbe97717da0b29ca26c64b20dfbda734a8ffb0560dbe8a4a1e10\": rpc error: code = NotFound desc = could not find container \"fba058facb69dbe97717da0b29ca26c64b20dfbda734a8ffb0560dbe8a4a1e10\": container with ID starting with fba058facb69dbe97717da0b29ca26c64b20dfbda734a8ffb0560dbe8a4a1e10 not found: ID does not exist"
Sep 11 11:52:13.622907 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.622860    2579 scope.go:117] "RemoveContainer" containerID="d57e4d1f6ee29011ce434f53ba3ebd911dcc91d53e2851725f505ffc317cf3bf"
Sep 11 11:52:13.623066 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.623049    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d57e4d1f6ee29011ce434f53ba3ebd911dcc91d53e2851725f505ffc317cf3bf"} err="failed to get container status \"d57e4d1f6ee29011ce434f53ba3ebd911dcc91d53e2851725f505ffc317cf3bf\": rpc error: code = NotFound desc = could not find container \"d57e4d1f6ee29011ce434f53ba3ebd911dcc91d53e2851725f505ffc317cf3bf\": container with ID starting with d57e4d1f6ee29011ce434f53ba3ebd911dcc91d53e2851725f505ffc317cf3bf not found: ID does not exist"
Sep 11 11:52:13.659893 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.659857    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 11:52:13.660167 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.660154    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08e855bb-cda7-4093-99d8-e5ce341100d5" containerName="prom-label-proxy"
Sep 11 11:52:13.660208 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.660169    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="08e855bb-cda7-4093-99d8-e5ce341100d5" containerName="prom-label-proxy"
Sep 11 11:52:13.660208 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.660180    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a803d13c-58b1-4c9f-940d-479f8ac5452a" containerName="kube-rbac-proxy"
Sep 11 11:52:13.660208 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.660186    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a803d13c-58b1-4c9f-940d-479f8ac5452a" containerName="kube-rbac-proxy"
Sep 11 11:52:13.660208 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.660197    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08e855bb-cda7-4093-99d8-e5ce341100d5" containerName="init-config-reloader"
Sep 11 11:52:13.660208 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.660203    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="08e855bb-cda7-4093-99d8-e5ce341100d5" containerName="init-config-reloader"
Sep 11 11:52:13.660352 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.660211    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a803d13c-58b1-4c9f-940d-479f8ac5452a" containerName="thanos-sidecar"
Sep 11 11:52:13.660352 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.660216    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a803d13c-58b1-4c9f-940d-479f8ac5452a" containerName="thanos-sidecar"
Sep 11 11:52:13.660352 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.660225    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08e855bb-cda7-4093-99d8-e5ce341100d5" containerName="config-reloader"
Sep 11 11:52:13.660352 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.660230    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="08e855bb-cda7-4093-99d8-e5ce341100d5" containerName="config-reloader"
Sep 11 11:52:13.660352 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.660236    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08e855bb-cda7-4093-99d8-e5ce341100d5" containerName="kube-rbac-proxy"
Sep 11 11:52:13.660352 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.660241    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="08e855bb-cda7-4093-99d8-e5ce341100d5" containerName="kube-rbac-proxy"
Sep 11 11:52:13.660352 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.660248    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a803d13c-58b1-4c9f-940d-479f8ac5452a" containerName="config-reloader"
Sep 11 11:52:13.660352 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.660253    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a803d13c-58b1-4c9f-940d-479f8ac5452a" containerName="config-reloader"
Sep 11 11:52:13.660352 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.660264    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a803d13c-58b1-4c9f-940d-479f8ac5452a" containerName="init-config-reloader"
Sep 11 11:52:13.660352 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.660269    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a803d13c-58b1-4c9f-940d-479f8ac5452a" containerName="init-config-reloader"
Sep 11 11:52:13.660352 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.660275    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a803d13c-58b1-4c9f-940d-479f8ac5452a" containerName="kube-rbac-proxy-web"
Sep 11 11:52:13.660352 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.660280    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a803d13c-58b1-4c9f-940d-479f8ac5452a" containerName="kube-rbac-proxy-web"
Sep 11 11:52:13.660352 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.660286    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a803d13c-58b1-4c9f-940d-479f8ac5452a" containerName="kube-rbac-proxy-thanos"
Sep 11 11:52:13.660352 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.660291    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a803d13c-58b1-4c9f-940d-479f8ac5452a" containerName="kube-rbac-proxy-thanos"
Sep 11 11:52:13.660352 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.660297    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08e855bb-cda7-4093-99d8-e5ce341100d5" containerName="kube-rbac-proxy-web"
Sep 11 11:52:13.660352 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.660302    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="08e855bb-cda7-4093-99d8-e5ce341100d5" containerName="kube-rbac-proxy-web"
Sep 11 11:52:13.660352 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.660308    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a803d13c-58b1-4c9f-940d-479f8ac5452a" containerName="prometheus"
Sep 11 11:52:13.660352 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.660313    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a803d13c-58b1-4c9f-940d-479f8ac5452a" containerName="prometheus"
Sep 11 11:52:13.660352 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.660319    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08e855bb-cda7-4093-99d8-e5ce341100d5" containerName="kube-rbac-proxy-metric"
Sep 11 11:52:13.660352 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.660325    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="08e855bb-cda7-4093-99d8-e5ce341100d5" containerName="kube-rbac-proxy-metric"
Sep 11 11:52:13.660352 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.660331    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08e855bb-cda7-4093-99d8-e5ce341100d5" containerName="alertmanager"
Sep 11 11:52:13.660352 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.660336    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="08e855bb-cda7-4093-99d8-e5ce341100d5" containerName="alertmanager"
Sep 11 11:52:13.661019 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.660382    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a803d13c-58b1-4c9f-940d-479f8ac5452a" containerName="prometheus"
Sep 11 11:52:13.661019 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.660410    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="08e855bb-cda7-4093-99d8-e5ce341100d5" containerName="kube-rbac-proxy"
Sep 11 11:52:13.661019 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.660417    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="08e855bb-cda7-4093-99d8-e5ce341100d5" containerName="kube-rbac-proxy-web"
Sep 11 11:52:13.661019 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.660424    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a803d13c-58b1-4c9f-940d-479f8ac5452a" containerName="thanos-sidecar"
Sep 11 11:52:13.661019 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.660429    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="08e855bb-cda7-4093-99d8-e5ce341100d5" containerName="alertmanager"
Sep 11 11:52:13.661019 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.660435    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="08e855bb-cda7-4093-99d8-e5ce341100d5" containerName="prom-label-proxy"
Sep 11 11:52:13.661019 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.660440    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a803d13c-58b1-4c9f-940d-479f8ac5452a" containerName="kube-rbac-proxy-thanos"
Sep 11 11:52:13.661019 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.660445    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="08e855bb-cda7-4093-99d8-e5ce341100d5" containerName="kube-rbac-proxy-metric"
Sep 11 11:52:13.661019 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.660451    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a803d13c-58b1-4c9f-940d-479f8ac5452a" containerName="config-reloader"
Sep 11 11:52:13.661019 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.660456    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="08e855bb-cda7-4093-99d8-e5ce341100d5" containerName="config-reloader"
Sep 11 11:52:13.661019 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.660461    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a803d13c-58b1-4c9f-940d-479f8ac5452a" containerName="kube-rbac-proxy-web"
Sep 11 11:52:13.661019 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.660466    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a803d13c-58b1-4c9f-940d-479f8ac5452a" containerName="kube-rbac-proxy"
Sep 11 11:52:13.665905 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.665885    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:52:13.676038 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.676012    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 11 11:52:13.677932 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.677914    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 11 11:52:13.678297 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.678273    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 11 11:52:13.678428 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.678303    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 11 11:52:13.681117 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.681100    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 11 11:52:13.688432 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.688414    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 11 11:52:13.688663 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.688647    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 11 11:52:13.689200 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.689184    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-zxjqj\""
Sep 11 11:52:13.689284 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.689201    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 11 11:52:13.689284 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.689189    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 11 11:52:13.689406 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.689371    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 11 11:52:13.689469 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.689456    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 11 11:52:13.689691 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.689675    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 11 11:52:13.689754 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.689678    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-d9v78es4a057a\""
Sep 11 11:52:13.690021 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.690007    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 11 11:52:13.690160 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.690144    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 11 11:52:13.700045 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.700021    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 11:52:13.800563 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.800525    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9e57c56f-2584-4328-b946-02b2ab076a97-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9e57c56f-2584-4328-b946-02b2ab076a97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:52:13.800563 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.800566    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9e57c56f-2584-4328-b946-02b2ab076a97-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9e57c56f-2584-4328-b946-02b2ab076a97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:52:13.800770 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.800591    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9e57c56f-2584-4328-b946-02b2ab076a97-config-out\") pod \"prometheus-k8s-0\" (UID: \"9e57c56f-2584-4328-b946-02b2ab076a97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:52:13.800770 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.800676    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9e57c56f-2584-4328-b946-02b2ab076a97-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9e57c56f-2584-4328-b946-02b2ab076a97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:52:13.800770 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.800713    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9e57c56f-2584-4328-b946-02b2ab076a97-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9e57c56f-2584-4328-b946-02b2ab076a97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:52:13.800770 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.800732    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9e57c56f-2584-4328-b946-02b2ab076a97-web-config\") pod \"prometheus-k8s-0\" (UID: \"9e57c56f-2584-4328-b946-02b2ab076a97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:52:13.800770 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.800749    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9e57c56f-2584-4328-b946-02b2ab076a97-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9e57c56f-2584-4328-b946-02b2ab076a97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:52:13.800939 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.800771    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9e57c56f-2584-4328-b946-02b2ab076a97-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9e57c56f-2584-4328-b946-02b2ab076a97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:52:13.800939 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.800791    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e57c56f-2584-4328-b946-02b2ab076a97-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9e57c56f-2584-4328-b946-02b2ab076a97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:52:13.800939 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.800806    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e57c56f-2584-4328-b946-02b2ab076a97-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9e57c56f-2584-4328-b946-02b2ab076a97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:52:13.800939 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.800820    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gq85h\" (UniqueName: \"kubernetes.io/projected/9e57c56f-2584-4328-b946-02b2ab076a97-kube-api-access-gq85h\") pod \"prometheus-k8s-0\" (UID: \"9e57c56f-2584-4328-b946-02b2ab076a97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:52:13.800939 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.800836    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9e57c56f-2584-4328-b946-02b2ab076a97-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9e57c56f-2584-4328-b946-02b2ab076a97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:52:13.800939 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.800900    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9e57c56f-2584-4328-b946-02b2ab076a97-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9e57c56f-2584-4328-b946-02b2ab076a97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:52:13.800939 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.800928    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e57c56f-2584-4328-b946-02b2ab076a97-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9e57c56f-2584-4328-b946-02b2ab076a97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:52:13.801208 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.800956    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9e57c56f-2584-4328-b946-02b2ab076a97-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9e57c56f-2584-4328-b946-02b2ab076a97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:52:13.801208 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.800975    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9e57c56f-2584-4328-b946-02b2ab076a97-config\") pod \"prometheus-k8s-0\" (UID: \"9e57c56f-2584-4328-b946-02b2ab076a97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:52:13.801208 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.800998    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9e57c56f-2584-4328-b946-02b2ab076a97-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9e57c56f-2584-4328-b946-02b2ab076a97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:52:13.801208 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.801015    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9e57c56f-2584-4328-b946-02b2ab076a97-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9e57c56f-2584-4328-b946-02b2ab076a97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:52:13.801208 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.801035    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9e57c56f-2584-4328-b946-02b2ab076a97-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9e57c56f-2584-4328-b946-02b2ab076a97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:52:13.801208 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.801056    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9e57c56f-2584-4328-b946-02b2ab076a97-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9e57c56f-2584-4328-b946-02b2ab076a97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:52:13.842845 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.842768    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:13.901634 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.901598    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9e57c56f-2584-4328-b946-02b2ab076a97-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9e57c56f-2584-4328-b946-02b2ab076a97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:52:13.901634 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.901636    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9e57c56f-2584-4328-b946-02b2ab076a97-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9e57c56f-2584-4328-b946-02b2ab076a97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:52:13.901839 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.901655    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9e57c56f-2584-4328-b946-02b2ab076a97-web-config\") pod \"prometheus-k8s-0\" (UID: \"9e57c56f-2584-4328-b946-02b2ab076a97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:52:13.901839 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.901673    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9e57c56f-2584-4328-b946-02b2ab076a97-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9e57c56f-2584-4328-b946-02b2ab076a97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:52:13.901839 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.901693    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9e57c56f-2584-4328-b946-02b2ab076a97-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9e57c56f-2584-4328-b946-02b2ab076a97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:52:13.901839 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.901712    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e57c56f-2584-4328-b946-02b2ab076a97-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9e57c56f-2584-4328-b946-02b2ab076a97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:52:13.901839 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.901730    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e57c56f-2584-4328-b946-02b2ab076a97-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9e57c56f-2584-4328-b946-02b2ab076a97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:52:13.901839 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.901749    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gq85h\" (UniqueName: \"kubernetes.io/projected/9e57c56f-2584-4328-b946-02b2ab076a97-kube-api-access-gq85h\") pod \"prometheus-k8s-0\" (UID: \"9e57c56f-2584-4328-b946-02b2ab076a97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:52:13.901839 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.901768    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9e57c56f-2584-4328-b946-02b2ab076a97-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9e57c56f-2584-4328-b946-02b2ab076a97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:52:13.901839 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.901798    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9e57c56f-2584-4328-b946-02b2ab076a97-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9e57c56f-2584-4328-b946-02b2ab076a97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:52:13.902344 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.902317    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e57c56f-2584-4328-b946-02b2ab076a97-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9e57c56f-2584-4328-b946-02b2ab076a97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:52:13.902465 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.902382    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9e57c56f-2584-4328-b946-02b2ab076a97-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9e57c56f-2584-4328-b946-02b2ab076a97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:52:13.902465 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.902436    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9e57c56f-2584-4328-b946-02b2ab076a97-config\") pod \"prometheus-k8s-0\" (UID: \"9e57c56f-2584-4328-b946-02b2ab076a97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:52:13.902465 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.902459    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9e57c56f-2584-4328-b946-02b2ab076a97-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9e57c56f-2584-4328-b946-02b2ab076a97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:52:13.902616 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.902483    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9e57c56f-2584-4328-b946-02b2ab076a97-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9e57c56f-2584-4328-b946-02b2ab076a97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:52:13.902616 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.902520    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9e57c56f-2584-4328-b946-02b2ab076a97-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9e57c56f-2584-4328-b946-02b2ab076a97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:52:13.902616 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.902525    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9e57c56f-2584-4328-b946-02b2ab076a97-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9e57c56f-2584-4328-b946-02b2ab076a97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:52:13.902616 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.902547    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9e57c56f-2584-4328-b946-02b2ab076a97-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9e57c56f-2584-4328-b946-02b2ab076a97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:52:13.902616 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.902567    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9e57c56f-2584-4328-b946-02b2ab076a97-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9e57c56f-2584-4328-b946-02b2ab076a97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:52:13.902616 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.902587    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9e57c56f-2584-4328-b946-02b2ab076a97-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9e57c56f-2584-4328-b946-02b2ab076a97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:52:13.902892 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.902624    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9e57c56f-2584-4328-b946-02b2ab076a97-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9e57c56f-2584-4328-b946-02b2ab076a97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:52:13.902892 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.902664    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9e57c56f-2584-4328-b946-02b2ab076a97-config-out\") pod \"prometheus-k8s-0\" (UID: \"9e57c56f-2584-4328-b946-02b2ab076a97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:52:13.902892 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.902737    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e57c56f-2584-4328-b946-02b2ab076a97-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9e57c56f-2584-4328-b946-02b2ab076a97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:52:13.902892 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.902775    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e57c56f-2584-4328-b946-02b2ab076a97-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9e57c56f-2584-4328-b946-02b2ab076a97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:52:13.903297 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.903268    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e57c56f-2584-4328-b946-02b2ab076a97-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9e57c56f-2584-4328-b946-02b2ab076a97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:52:13.904354 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.903310    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9e57c56f-2584-4328-b946-02b2ab076a97-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9e57c56f-2584-4328-b946-02b2ab076a97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:52:13.904354 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.903899    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9e57c56f-2584-4328-b946-02b2ab076a97-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9e57c56f-2584-4328-b946-02b2ab076a97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:52:13.905755 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.905700    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9e57c56f-2584-4328-b946-02b2ab076a97-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9e57c56f-2584-4328-b946-02b2ab076a97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:52:13.906026 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.906000    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9e57c56f-2584-4328-b946-02b2ab076a97-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9e57c56f-2584-4328-b946-02b2ab076a97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:52:13.906307 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.906248    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9e57c56f-2584-4328-b946-02b2ab076a97-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9e57c56f-2584-4328-b946-02b2ab076a97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:52:13.906843 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.906729    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9e57c56f-2584-4328-b946-02b2ab076a97-config-out\") pod \"prometheus-k8s-0\" (UID: \"9e57c56f-2584-4328-b946-02b2ab076a97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:52:13.906843 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.906806    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9e57c56f-2584-4328-b946-02b2ab076a97-web-config\") pod \"prometheus-k8s-0\" (UID: \"9e57c56f-2584-4328-b946-02b2ab076a97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:52:13.906991 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.906934    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9e57c56f-2584-4328-b946-02b2ab076a97-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9e57c56f-2584-4328-b946-02b2ab076a97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:52:13.907047 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.906993    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9e57c56f-2584-4328-b946-02b2ab076a97-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9e57c56f-2584-4328-b946-02b2ab076a97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:52:13.907267 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.907248    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/9e57c56f-2584-4328-b946-02b2ab076a97-config\") pod \"prometheus-k8s-0\" (UID: \"9e57c56f-2584-4328-b946-02b2ab076a97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:52:13.907584 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.907563    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9e57c56f-2584-4328-b946-02b2ab076a97-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9e57c56f-2584-4328-b946-02b2ab076a97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:52:13.907928 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.907911    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9e57c56f-2584-4328-b946-02b2ab076a97-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9e57c56f-2584-4328-b946-02b2ab076a97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:52:13.908255 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.908237    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9e57c56f-2584-4328-b946-02b2ab076a97-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9e57c56f-2584-4328-b946-02b2ab076a97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:52:13.908300 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.908245    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9e57c56f-2584-4328-b946-02b2ab076a97-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9e57c56f-2584-4328-b946-02b2ab076a97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:52:13.911195 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.911178    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gq85h\" (UniqueName: \"kubernetes.io/projected/9e57c56f-2584-4328-b946-02b2ab076a97-kube-api-access-gq85h\") pod \"prometheus-k8s-0\" (UID: \"9e57c56f-2584-4328-b946-02b2ab076a97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:52:13.975175 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:13.975134    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:52:14.262152 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:14.262102    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 11:52:14.264444 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:52:14.264413    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e57c56f_2584_4328_b946_02b2ab076a97.slice/crio-82e06d248627ed6bc79e88f3ac587c78854c5716a2cdb402b588fbbee67cf994 WatchSource:0}: Error finding container 82e06d248627ed6bc79e88f3ac587c78854c5716a2cdb402b588fbbee67cf994: Status 404 returned error can't find the container with id 82e06d248627ed6bc79e88f3ac587c78854c5716a2cdb402b588fbbee67cf994
Sep 11 11:52:14.564508 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:14.564427    2579 generic.go:356] "Generic (PLEG): container finished" podID="9e57c56f-2584-4328-b946-02b2ab076a97" containerID="ca930e8325b14d2c9650e01ee5ac9dd39b1dce27c7bfb24c5e23dbcf01389a52" exitCode=0
Sep 11 11:52:14.564921 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:14.564527    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9e57c56f-2584-4328-b946-02b2ab076a97","Type":"ContainerDied","Data":"ca930e8325b14d2c9650e01ee5ac9dd39b1dce27c7bfb24c5e23dbcf01389a52"}
Sep 11 11:52:14.564921 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:14.564613    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9e57c56f-2584-4328-b946-02b2ab076a97","Type":"ContainerStarted","Data":"82e06d248627ed6bc79e88f3ac587c78854c5716a2cdb402b588fbbee67cf994"}
Sep 11 11:52:14.574738 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:14.574707    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a803d13c-58b1-4c9f-940d-479f8ac5452a" path="/var/lib/kubelet/pods/a803d13c-58b1-4c9f-940d-479f8ac5452a/volumes"
Sep 11 11:52:15.573119 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:15.573082    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9e57c56f-2584-4328-b946-02b2ab076a97","Type":"ContainerStarted","Data":"fe795771cac4bc283b6062bf8a48496016677376015de038ef7dfd166eed4ee9"}
Sep 11 11:52:15.573504 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:15.573152    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9e57c56f-2584-4328-b946-02b2ab076a97","Type":"ContainerStarted","Data":"31b25a3fcaae6ba858fda9e45b7573e008ff3340e69f465f857ef8ab1943023d"}
Sep 11 11:52:15.573504 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:15.573178    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9e57c56f-2584-4328-b946-02b2ab076a97","Type":"ContainerStarted","Data":"1017faf5c1c155c4fc64fb3566615181eedcd157dcef7637a288822f3c8f816f"}
Sep 11 11:52:15.573504 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:15.573204    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9e57c56f-2584-4328-b946-02b2ab076a97","Type":"ContainerStarted","Data":"41c58de09571f128a879c6a035e1e5789f54a1328a0007fbd966ac1913f8a32d"}
Sep 11 11:52:15.573504 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:15.573229    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9e57c56f-2584-4328-b946-02b2ab076a97","Type":"ContainerStarted","Data":"15ccaa9bc0680285046e47f5fd817c6b113575fd8e03287208c5b39dac42885c"}
Sep 11 11:52:15.573504 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:15.573252    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9e57c56f-2584-4328-b946-02b2ab076a97","Type":"ContainerStarted","Data":"ea829a00f50bd856e29a611ab69495a9ddea74fb1c6c1acc156524f4ace3ca51"}
Sep 11 11:52:15.606865 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:15.606793    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.584474632 podStartE2EDuration="2.606769155s" totalImagesPullingTime="0s" totalInitContainerRuntime="22.294523ms" isStatefulPod=false podCreationTimestamp="2026-09-11 11:52:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:52:15.604215542 +0000 UTC m=+239.501657310" watchObservedRunningTime="2026-09-11 11:52:15.606769155 +0000 UTC m=+239.504210901"
Sep 11 11:52:15.723687 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:15.723651    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:15.796315 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:15.796278    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:18.976019 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:18.975970    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:52:22.719643 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:22.719612    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:22.820986 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:22.820948    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:22.902642 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:22.902600    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:24.431940 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:24.431903    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:24.531936 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:24.531891    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:50.294346 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:50.294256    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:50.381807 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:52:50.381766    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:13.976316 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:53:13.976280    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:53:13.991706 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:53:13.991680    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:53:14.419781 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:53:14.419737    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:14.824743 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:53:14.824677    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:53:16.443850 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:53:16.443819    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-ch96n_1f0f9fa5-cbd1-468b-ae81-82b48c101d37/console-operator/2.log"
Sep 11 11:53:16.447869 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:53:16.447844    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-ch96n_1f0f9fa5-cbd1-468b-ae81-82b48c101d37/console-operator/2.log"
Sep 11 11:53:16.455148 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:53:16.455120    2579 kubelet.go:1648] "Image garbage collection succeeded"
Sep 11 11:53:41.397384 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:53:41.397341    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:41.527488 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:53:41.527456    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:04.167047 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:04.167014    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:04.522511 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:04.522425    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:08.563631 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:08.563594    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:08.660224 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:08.660189    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:10.411383 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:10.411346    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:10.510219 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:10.510182    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:10.978299 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:10.978267    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:11.115247 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:11.115220    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:12.245510 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:12.245473    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:12.879959 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:12.879929    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:14.756766 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:14.756730    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:14.825066 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:14.825033    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:20.123530 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:20.123494    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:20.224199 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:20.224169    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:21.628020 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:21.627982    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:21.860584 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:21.860547    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:21.953714 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:21.953625    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:28.825017 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:28.824979    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:28.918009 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:28.917971    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:30.636646 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:30.636598    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:45.628894 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:45.628858    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:45.733118 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:45.733076    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:46.788871 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:46.788829    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:46.896707 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:46.896670    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:47.443573 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:47.443537    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:47.787900 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:47.787805    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:47.881871 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:47.881836    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:47.953118 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:47.953078    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7fb8846894-drb98"]
Sep 11 11:54:47.956466 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:47.956443    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7fb8846894-drb98"
Sep 11 11:54:47.959340 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:47.959311    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 11 11:54:47.959476 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:47.959380    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 11 11:54:47.960530 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:47.960513    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 11 11:54:47.961653 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:47.961635    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 11 11:54:47.961729 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:47.961709    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-47w2g\""
Sep 11 11:54:47.984145 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:47.984120    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7fb8846894-drb98"]
Sep 11 11:54:48.081521 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:48.081423    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b5a20575-88da-4ed8-ab26-487b7ef2ae80-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7fb8846894-drb98\" (UID: \"b5a20575-88da-4ed8-ab26-487b7ef2ae80\") " pod="opendatahub/opendatahub-operator-controller-manager-7fb8846894-drb98"
Sep 11 11:54:48.081521 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:48.081476    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b5a20575-88da-4ed8-ab26-487b7ef2ae80-manifests\") pod \"opendatahub-operator-controller-manager-7fb8846894-drb98\" (UID: \"b5a20575-88da-4ed8-ab26-487b7ef2ae80\") " pod="opendatahub/opendatahub-operator-controller-manager-7fb8846894-drb98"
Sep 11 11:54:48.081521 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:48.081507    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b5a20575-88da-4ed8-ab26-487b7ef2ae80-tmp\") pod \"opendatahub-operator-controller-manager-7fb8846894-drb98\" (UID: \"b5a20575-88da-4ed8-ab26-487b7ef2ae80\") " pod="opendatahub/opendatahub-operator-controller-manager-7fb8846894-drb98"
Sep 11 11:54:48.081787 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:48.081598    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrkw8\" (UniqueName: \"kubernetes.io/projected/b5a20575-88da-4ed8-ab26-487b7ef2ae80-kube-api-access-xrkw8\") pod \"opendatahub-operator-controller-manager-7fb8846894-drb98\" (UID: \"b5a20575-88da-4ed8-ab26-487b7ef2ae80\") " pod="opendatahub/opendatahub-operator-controller-manager-7fb8846894-drb98"
Sep 11 11:54:48.081787 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:48.081647    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b5a20575-88da-4ed8-ab26-487b7ef2ae80-webhook-cert\") pod \"opendatahub-operator-controller-manager-7fb8846894-drb98\" (UID: \"b5a20575-88da-4ed8-ab26-487b7ef2ae80\") " pod="opendatahub/opendatahub-operator-controller-manager-7fb8846894-drb98"
Sep 11 11:54:48.182659 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:48.182616    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b5a20575-88da-4ed8-ab26-487b7ef2ae80-webhook-cert\") pod \"opendatahub-operator-controller-manager-7fb8846894-drb98\" (UID: \"b5a20575-88da-4ed8-ab26-487b7ef2ae80\") " pod="opendatahub/opendatahub-operator-controller-manager-7fb8846894-drb98"
Sep 11 11:54:48.182840 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:48.182680    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b5a20575-88da-4ed8-ab26-487b7ef2ae80-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7fb8846894-drb98\" (UID: \"b5a20575-88da-4ed8-ab26-487b7ef2ae80\") " pod="opendatahub/opendatahub-operator-controller-manager-7fb8846894-drb98"
Sep 11 11:54:48.182840 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:48.182712    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b5a20575-88da-4ed8-ab26-487b7ef2ae80-manifests\") pod \"opendatahub-operator-controller-manager-7fb8846894-drb98\" (UID: \"b5a20575-88da-4ed8-ab26-487b7ef2ae80\") " pod="opendatahub/opendatahub-operator-controller-manager-7fb8846894-drb98"
Sep 11 11:54:48.182840 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:48.182742    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b5a20575-88da-4ed8-ab26-487b7ef2ae80-tmp\") pod \"opendatahub-operator-controller-manager-7fb8846894-drb98\" (UID: \"b5a20575-88da-4ed8-ab26-487b7ef2ae80\") " pod="opendatahub/opendatahub-operator-controller-manager-7fb8846894-drb98"
Sep 11 11:54:48.182840 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:48.182770    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrkw8\" (UniqueName: \"kubernetes.io/projected/b5a20575-88da-4ed8-ab26-487b7ef2ae80-kube-api-access-xrkw8\") pod \"opendatahub-operator-controller-manager-7fb8846894-drb98\" (UID: \"b5a20575-88da-4ed8-ab26-487b7ef2ae80\") " pod="opendatahub/opendatahub-operator-controller-manager-7fb8846894-drb98"
Sep 11 11:54:48.183237 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:48.183211    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b5a20575-88da-4ed8-ab26-487b7ef2ae80-manifests\") pod \"opendatahub-operator-controller-manager-7fb8846894-drb98\" (UID: \"b5a20575-88da-4ed8-ab26-487b7ef2ae80\") " pod="opendatahub/opendatahub-operator-controller-manager-7fb8846894-drb98"
Sep 11 11:54:48.184963 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:48.184934    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b5a20575-88da-4ed8-ab26-487b7ef2ae80-tmp\") pod \"opendatahub-operator-controller-manager-7fb8846894-drb98\" (UID: \"b5a20575-88da-4ed8-ab26-487b7ef2ae80\") " pod="opendatahub/opendatahub-operator-controller-manager-7fb8846894-drb98"
Sep 11 11:54:48.185274 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:48.185252    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b5a20575-88da-4ed8-ab26-487b7ef2ae80-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7fb8846894-drb98\" (UID: \"b5a20575-88da-4ed8-ab26-487b7ef2ae80\") " pod="opendatahub/opendatahub-operator-controller-manager-7fb8846894-drb98"
Sep 11 11:54:48.185366 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:48.185349    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b5a20575-88da-4ed8-ab26-487b7ef2ae80-webhook-cert\") pod \"opendatahub-operator-controller-manager-7fb8846894-drb98\" (UID: \"b5a20575-88da-4ed8-ab26-487b7ef2ae80\") " pod="opendatahub/opendatahub-operator-controller-manager-7fb8846894-drb98"
Sep 11 11:54:48.198536 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:48.198513    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrkw8\" (UniqueName: \"kubernetes.io/projected/b5a20575-88da-4ed8-ab26-487b7ef2ae80-kube-api-access-xrkw8\") pod \"opendatahub-operator-controller-manager-7fb8846894-drb98\" (UID: \"b5a20575-88da-4ed8-ab26-487b7ef2ae80\") " pod="opendatahub/opendatahub-operator-controller-manager-7fb8846894-drb98"
Sep 11 11:54:48.267230 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:48.267192    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7fb8846894-drb98"
Sep 11 11:54:48.303526 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:48.303489    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:48.389928 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:48.389892    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:48.492702 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:48.492666    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:48.554968 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:48.554936    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7fb8846894-drb98"]
Sep 11 11:54:48.558295 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:54:48.558260    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5a20575_88da_4ed8_ab26_487b7ef2ae80.slice/crio-b1e93dc92a3d049bc384c9b1e496735cde83523628c3202d54faf2c7666da110 WatchSource:0}: Error finding container b1e93dc92a3d049bc384c9b1e496735cde83523628c3202d54faf2c7666da110: Status 404 returned error can't find the container with id b1e93dc92a3d049bc384c9b1e496735cde83523628c3202d54faf2c7666da110
Sep 11 11:54:48.560117 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:48.560098    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 11:54:48.593140 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:48.593101    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:48.697892 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:48.697804    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:48.809178 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:48.809142    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:48.935515 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:48.935479    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:49.195852 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:49.195817    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7fb8846894-drb98" event={"ID":"b5a20575-88da-4ed8-ab26-487b7ef2ae80","Type":"ContainerStarted","Data":"b1e93dc92a3d049bc384c9b1e496735cde83523628c3202d54faf2c7666da110"}
Sep 11 11:54:52.211212 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:52.211175    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7fb8846894-drb98" event={"ID":"b5a20575-88da-4ed8-ab26-487b7ef2ae80","Type":"ContainerStarted","Data":"e72864e38a4edc1b291b4862e11b4cef6d5a24a18269ba1aad2cccb5b25efe62"}
Sep 11 11:54:53.217843 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:53.217807    2579 generic.go:356] "Generic (PLEG): container finished" podID="b5a20575-88da-4ed8-ab26-487b7ef2ae80" containerID="e72864e38a4edc1b291b4862e11b4cef6d5a24a18269ba1aad2cccb5b25efe62" exitCode=0
Sep 11 11:54:53.218238 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:53.217890    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7fb8846894-drb98" event={"ID":"b5a20575-88da-4ed8-ab26-487b7ef2ae80","Type":"ContainerDied","Data":"e72864e38a4edc1b291b4862e11b4cef6d5a24a18269ba1aad2cccb5b25efe62"}
Sep 11 11:54:53.911512 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:53.911467    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:54.224214 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:54.224180    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7fb8846894-drb98" event={"ID":"b5a20575-88da-4ed8-ab26-487b7ef2ae80","Type":"ContainerStarted","Data":"5bbcc8c053c014a3f13451864661323a59c638da29b1b306608f5304eaf6580c"}
Sep 11 11:54:54.224602 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:54.224317    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7fb8846894-drb98"
Sep 11 11:54:54.252055 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:54.251991    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7fb8846894-drb98" podStartSLOduration=2.936567769 podStartE2EDuration="7.251974625s" totalImagesPullingTime="3.259439653s" totalInitContainerRuntime="1.055967203s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:54:47 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:54:54.25009509 +0000 UTC m=+398.147536836" watchObservedRunningTime="2026-09-11 11:54:54.251974625 +0000 UTC m=+398.149416371"
Sep 11 11:54:55.853214 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:55.853180    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:55.913564 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:55.913526    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:59.918332 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:54:59.918294    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:55:00.033482 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:00.033432    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:55:05.231248 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:05.231213    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7fb8846894-drb98"
Sep 11 11:55:07.643379 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:07.643338    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:55:07.841740 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:07.841707    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:55:13.461535 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:13.461494    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:55:13.565474 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:13.565442    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-57dff8dc5-xn758"]
Sep 11 11:55:13.568729 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:13.568704    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-57dff8dc5-xn758"
Sep 11 11:55:13.571228 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:13.571197    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-gqrk7\""
Sep 11 11:55:13.572154 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:13.572112    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 11 11:55:13.572275 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:13.572164    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 11 11:55:13.581313 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:13.581285    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-57dff8dc5-xn758"]
Sep 11 11:55:13.615843 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:13.615797    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47d3ab09-6e98-4185-8c2c-4ee78873af56-tls-certs\") pod \"kube-auth-proxy-57dff8dc5-xn758\" (UID: \"47d3ab09-6e98-4185-8c2c-4ee78873af56\") " pod="openshift-ingress/kube-auth-proxy-57dff8dc5-xn758"
Sep 11 11:55:13.616039 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:13.615931    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/47d3ab09-6e98-4185-8c2c-4ee78873af56-tmp\") pod \"kube-auth-proxy-57dff8dc5-xn758\" (UID: \"47d3ab09-6e98-4185-8c2c-4ee78873af56\") " pod="openshift-ingress/kube-auth-proxy-57dff8dc5-xn758"
Sep 11 11:55:13.616039 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:13.615965    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bl6wm\" (UniqueName: \"kubernetes.io/projected/47d3ab09-6e98-4185-8c2c-4ee78873af56-kube-api-access-bl6wm\") pod \"kube-auth-proxy-57dff8dc5-xn758\" (UID: \"47d3ab09-6e98-4185-8c2c-4ee78873af56\") " pod="openshift-ingress/kube-auth-proxy-57dff8dc5-xn758"
Sep 11 11:55:13.717274 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:13.717173    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/47d3ab09-6e98-4185-8c2c-4ee78873af56-tmp\") pod \"kube-auth-proxy-57dff8dc5-xn758\" (UID: \"47d3ab09-6e98-4185-8c2c-4ee78873af56\") " pod="openshift-ingress/kube-auth-proxy-57dff8dc5-xn758"
Sep 11 11:55:13.717274 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:13.717216    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bl6wm\" (UniqueName: \"kubernetes.io/projected/47d3ab09-6e98-4185-8c2c-4ee78873af56-kube-api-access-bl6wm\") pod \"kube-auth-proxy-57dff8dc5-xn758\" (UID: \"47d3ab09-6e98-4185-8c2c-4ee78873af56\") " pod="openshift-ingress/kube-auth-proxy-57dff8dc5-xn758"
Sep 11 11:55:13.717274 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:13.717245    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47d3ab09-6e98-4185-8c2c-4ee78873af56-tls-certs\") pod \"kube-auth-proxy-57dff8dc5-xn758\" (UID: \"47d3ab09-6e98-4185-8c2c-4ee78873af56\") " pod="openshift-ingress/kube-auth-proxy-57dff8dc5-xn758"
Sep 11 11:55:13.719504 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:13.719477    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/47d3ab09-6e98-4185-8c2c-4ee78873af56-tmp\") pod \"kube-auth-proxy-57dff8dc5-xn758\" (UID: \"47d3ab09-6e98-4185-8c2c-4ee78873af56\") " pod="openshift-ingress/kube-auth-proxy-57dff8dc5-xn758"
Sep 11 11:55:13.719699 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:13.719682    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47d3ab09-6e98-4185-8c2c-4ee78873af56-tls-certs\") pod \"kube-auth-proxy-57dff8dc5-xn758\" (UID: \"47d3ab09-6e98-4185-8c2c-4ee78873af56\") " pod="openshift-ingress/kube-auth-proxy-57dff8dc5-xn758"
Sep 11 11:55:13.726571 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:13.726548    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bl6wm\" (UniqueName: \"kubernetes.io/projected/47d3ab09-6e98-4185-8c2c-4ee78873af56-kube-api-access-bl6wm\") pod \"kube-auth-proxy-57dff8dc5-xn758\" (UID: \"47d3ab09-6e98-4185-8c2c-4ee78873af56\") " pod="openshift-ingress/kube-auth-proxy-57dff8dc5-xn758"
Sep 11 11:55:13.880840 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:13.880804    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-57dff8dc5-xn758"
Sep 11 11:55:14.159455 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:14.159422    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-57dff8dc5-xn758"]
Sep 11 11:55:14.159942 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:55:14.159918    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47d3ab09_6e98_4185_8c2c_4ee78873af56.slice/crio-cd04bc0f56629900eba018aa586fc43581c1baeedf4db41fb0176e098b577483 WatchSource:0}: Error finding container cd04bc0f56629900eba018aa586fc43581c1baeedf4db41fb0176e098b577483: Status 404 returned error can't find the container with id cd04bc0f56629900eba018aa586fc43581c1baeedf4db41fb0176e098b577483
Sep 11 11:55:14.311972 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:14.311929    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-57dff8dc5-xn758" event={"ID":"47d3ab09-6e98-4185-8c2c-4ee78873af56","Type":"ContainerStarted","Data":"cd04bc0f56629900eba018aa586fc43581c1baeedf4db41fb0176e098b577483"}
Sep 11 11:55:15.021503 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:15.021437    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:55:15.130077 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:15.129937    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:55:15.199870 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:15.199832    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:55:16.219664 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:16.219624    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:55:16.898717 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:16.898695    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 11 11:55:17.326984 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:17.326897    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-57dff8dc5-xn758" event={"ID":"47d3ab09-6e98-4185-8c2c-4ee78873af56","Type":"ContainerStarted","Data":"74ef5af0daeb381ed7fbb9f82dc7c1b3e238b6eb48f6e575eae66023146fd2de"}
Sep 11 11:55:17.354888 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:17.354833    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-57dff8dc5-xn758" podStartSLOduration=1.621235176 podStartE2EDuration="4.354816452s" totalImagesPullingTime="2.733581276s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:55:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:55:17.353094703 +0000 UTC m=+421.250536449" watchObservedRunningTime="2026-09-11 11:55:17.354816452 +0000 UTC m=+421.252258239"
Sep 11 11:55:18.695230 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:18.695175    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-mvbmv"]
Sep 11 11:55:18.698854 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:18.698836    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-mvbmv"
Sep 11 11:55:18.701567 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:18.701544    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Sep 11 11:55:18.701567 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:18.701563    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-kzqmd\""
Sep 11 11:55:18.713329 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:18.713297    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-mvbmv"]
Sep 11 11:55:18.743120 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:18.743079    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:55:18.763544 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:18.763508    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzpdv\" (UniqueName: \"kubernetes.io/projected/59b9266e-e066-4bfc-8a4e-505d6fb6f7dd-kube-api-access-mzpdv\") pod \"odh-model-controller-7998c67dc4-mvbmv\" (UID: \"59b9266e-e066-4bfc-8a4e-505d6fb6f7dd\") " pod="opendatahub/odh-model-controller-7998c67dc4-mvbmv"
Sep 11 11:55:18.763544 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:18.763546    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/59b9266e-e066-4bfc-8a4e-505d6fb6f7dd-cert\") pod \"odh-model-controller-7998c67dc4-mvbmv\" (UID: \"59b9266e-e066-4bfc-8a4e-505d6fb6f7dd\") " pod="opendatahub/odh-model-controller-7998c67dc4-mvbmv"
Sep 11 11:55:18.833907 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:18.833872    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:55:18.864498 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:18.864460    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzpdv\" (UniqueName: \"kubernetes.io/projected/59b9266e-e066-4bfc-8a4e-505d6fb6f7dd-kube-api-access-mzpdv\") pod \"odh-model-controller-7998c67dc4-mvbmv\" (UID: \"59b9266e-e066-4bfc-8a4e-505d6fb6f7dd\") " pod="opendatahub/odh-model-controller-7998c67dc4-mvbmv"
Sep 11 11:55:18.864498 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:18.864500    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/59b9266e-e066-4bfc-8a4e-505d6fb6f7dd-cert\") pod \"odh-model-controller-7998c67dc4-mvbmv\" (UID: \"59b9266e-e066-4bfc-8a4e-505d6fb6f7dd\") " pod="opendatahub/odh-model-controller-7998c67dc4-mvbmv"
Sep 11 11:55:18.864741 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:55:18.864624    2579 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Sep 11 11:55:18.864741 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:55:18.864688    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59b9266e-e066-4bfc-8a4e-505d6fb6f7dd-cert podName:59b9266e-e066-4bfc-8a4e-505d6fb6f7dd nodeName:}" failed. No retries permitted until 2026-09-11 11:55:19.36466829 +0000 UTC m=+423.262110013 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/59b9266e-e066-4bfc-8a4e-505d6fb6f7dd-cert") pod "odh-model-controller-7998c67dc4-mvbmv" (UID: "59b9266e-e066-4bfc-8a4e-505d6fb6f7dd") : secret "odh-model-controller-webhook-cert" not found
Sep 11 11:55:18.874069 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:18.874043    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzpdv\" (UniqueName: \"kubernetes.io/projected/59b9266e-e066-4bfc-8a4e-505d6fb6f7dd-kube-api-access-mzpdv\") pod \"odh-model-controller-7998c67dc4-mvbmv\" (UID: \"59b9266e-e066-4bfc-8a4e-505d6fb6f7dd\") " pod="opendatahub/odh-model-controller-7998c67dc4-mvbmv"
Sep 11 11:55:19.000263 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:19.000184    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:55:19.098024 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:19.097989    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:55:19.369951 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:19.369920    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/59b9266e-e066-4bfc-8a4e-505d6fb6f7dd-cert\") pod \"odh-model-controller-7998c67dc4-mvbmv\" (UID: \"59b9266e-e066-4bfc-8a4e-505d6fb6f7dd\") " pod="opendatahub/odh-model-controller-7998c67dc4-mvbmv"
Sep 11 11:55:19.372418 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:19.372381    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/59b9266e-e066-4bfc-8a4e-505d6fb6f7dd-cert\") pod \"odh-model-controller-7998c67dc4-mvbmv\" (UID: \"59b9266e-e066-4bfc-8a4e-505d6fb6f7dd\") " pod="opendatahub/odh-model-controller-7998c67dc4-mvbmv"
Sep 11 11:55:19.419652 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:19.419621    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:55:19.443535 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:19.443498    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-4zb6j"]
Sep 11 11:55:19.446955 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:19.446937    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-4zb6j"
Sep 11 11:55:19.450050 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:19.450030    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Sep 11 11:55:19.457629 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:19.457602    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-4zb6j"]
Sep 11 11:55:19.471135 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:19.471104    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5348a878-29ad-49e6-a1cf-aab044f10e14-tls-certs\") pod \"model-serving-api-54d89bd5fc-4zb6j\" (UID: \"5348a878-29ad-49e6-a1cf-aab044f10e14\") " pod="opendatahub/model-serving-api-54d89bd5fc-4zb6j"
Sep 11 11:55:19.471283 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:19.471145    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24gm4\" (UniqueName: \"kubernetes.io/projected/5348a878-29ad-49e6-a1cf-aab044f10e14-kube-api-access-24gm4\") pod \"model-serving-api-54d89bd5fc-4zb6j\" (UID: \"5348a878-29ad-49e6-a1cf-aab044f10e14\") " pod="opendatahub/model-serving-api-54d89bd5fc-4zb6j"
Sep 11 11:55:19.571528 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:19.571488    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24gm4\" (UniqueName: \"kubernetes.io/projected/5348a878-29ad-49e6-a1cf-aab044f10e14-kube-api-access-24gm4\") pod \"model-serving-api-54d89bd5fc-4zb6j\" (UID: \"5348a878-29ad-49e6-a1cf-aab044f10e14\") " pod="opendatahub/model-serving-api-54d89bd5fc-4zb6j"
Sep 11 11:55:19.571705 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:19.571585    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5348a878-29ad-49e6-a1cf-aab044f10e14-tls-certs\") pod \"model-serving-api-54d89bd5fc-4zb6j\" (UID: \"5348a878-29ad-49e6-a1cf-aab044f10e14\") " pod="opendatahub/model-serving-api-54d89bd5fc-4zb6j"
Sep 11 11:55:19.571705 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:55:19.571674    2579 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Sep 11 11:55:19.571794 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:55:19.571732    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5348a878-29ad-49e6-a1cf-aab044f10e14-tls-certs podName:5348a878-29ad-49e6-a1cf-aab044f10e14 nodeName:}" failed. No retries permitted until 2026-09-11 11:55:20.071714519 +0000 UTC m=+423.969156259 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/5348a878-29ad-49e6-a1cf-aab044f10e14-tls-certs") pod "model-serving-api-54d89bd5fc-4zb6j" (UID: "5348a878-29ad-49e6-a1cf-aab044f10e14") : secret "model-serving-api-tls" not found
Sep 11 11:55:19.581240 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:19.581204    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-24gm4\" (UniqueName: \"kubernetes.io/projected/5348a878-29ad-49e6-a1cf-aab044f10e14-kube-api-access-24gm4\") pod \"model-serving-api-54d89bd5fc-4zb6j\" (UID: \"5348a878-29ad-49e6-a1cf-aab044f10e14\") " pod="opendatahub/model-serving-api-54d89bd5fc-4zb6j"
Sep 11 11:55:19.608852 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:19.608817    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-mvbmv"
Sep 11 11:55:19.784792 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:19.784751    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:55:19.883576 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:19.883540    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-mvbmv"]
Sep 11 11:55:19.886851 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:55:19.886823    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59b9266e_e066_4bfc_8a4e_505d6fb6f7dd.slice/crio-234da16483bb8955b0d91d83c2fdc1bda53866561927fa0edcf22c50e40ccbbc WatchSource:0}: Error finding container 234da16483bb8955b0d91d83c2fdc1bda53866561927fa0edcf22c50e40ccbbc: Status 404 returned error can't find the container with id 234da16483bb8955b0d91d83c2fdc1bda53866561927fa0edcf22c50e40ccbbc
Sep 11 11:55:20.075652 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:20.075558    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5348a878-29ad-49e6-a1cf-aab044f10e14-tls-certs\") pod \"model-serving-api-54d89bd5fc-4zb6j\" (UID: \"5348a878-29ad-49e6-a1cf-aab044f10e14\") " pod="opendatahub/model-serving-api-54d89bd5fc-4zb6j"
Sep 11 11:55:20.078013 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:20.077985    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5348a878-29ad-49e6-a1cf-aab044f10e14-tls-certs\") pod \"model-serving-api-54d89bd5fc-4zb6j\" (UID: \"5348a878-29ad-49e6-a1cf-aab044f10e14\") " pod="opendatahub/model-serving-api-54d89bd5fc-4zb6j"
Sep 11 11:55:20.340899 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:20.340806    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-mvbmv" event={"ID":"59b9266e-e066-4bfc-8a4e-505d6fb6f7dd","Type":"ContainerStarted","Data":"234da16483bb8955b0d91d83c2fdc1bda53866561927fa0edcf22c50e40ccbbc"}
Sep 11 11:55:20.358180 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:20.358136    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-4zb6j"
Sep 11 11:55:20.631808 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:20.631728    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-4zb6j"]
Sep 11 11:55:20.634450 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:55:20.634418    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5348a878_29ad_49e6_a1cf_aab044f10e14.slice/crio-84e6be4d5f4eebe5abb628516192b22344f89465c215e88084875b580a5878d1 WatchSource:0}: Error finding container 84e6be4d5f4eebe5abb628516192b22344f89465c215e88084875b580a5878d1: Status 404 returned error can't find the container with id 84e6be4d5f4eebe5abb628516192b22344f89465c215e88084875b580a5878d1
Sep 11 11:55:20.942715 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:20.942620    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:55:21.080668 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:21.080632    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:55:21.348198 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:21.348160    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-4zb6j" event={"ID":"5348a878-29ad-49e6-a1cf-aab044f10e14","Type":"ContainerStarted","Data":"84e6be4d5f4eebe5abb628516192b22344f89465c215e88084875b580a5878d1"}
Sep 11 11:55:21.757864 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:21.757732    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:55:22.071933 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:22.071836    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:55:22.194921 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:22.194871    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:55:23.359740 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:23.359704    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-mvbmv" event={"ID":"59b9266e-e066-4bfc-8a4e-505d6fb6f7dd","Type":"ContainerStarted","Data":"e60a644876cd15a51b3816a1346a89f222bf54bc7e654c773c7c4b4dab51bf78"}
Sep 11 11:55:23.360215 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:23.359770    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-mvbmv"
Sep 11 11:55:23.361383 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:23.361346    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-4zb6j" event={"ID":"5348a878-29ad-49e6-a1cf-aab044f10e14","Type":"ContainerStarted","Data":"75651bbccacdc6f6a296f68f5db33bbca56b3152f12ec5e77b2cead4a4a7e6c3"}
Sep 11 11:55:23.361459 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:23.361419    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-4zb6j"
Sep 11 11:55:23.378594 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:23.378531    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-mvbmv" podStartSLOduration=2.475989026 podStartE2EDuration="5.378511174s" totalImagesPullingTime="2.902522148s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:55:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:55:23.376487925 +0000 UTC m=+427.273929671" watchObservedRunningTime="2026-09-11 11:55:23.378511174 +0000 UTC m=+427.275952921"
Sep 11 11:55:26.162150 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:26.162093    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-4zb6j" podStartSLOduration=5.010254806 podStartE2EDuration="7.162076327s" totalImagesPullingTime="2.151821521s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:55:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:55:23.406950548 +0000 UTC m=+427.304392293" watchObservedRunningTime="2026-09-11 11:55:26.162076327 +0000 UTC m=+430.059518069"
Sep 11 11:55:26.163321 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:26.163302    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-zc7qw"]
Sep 11 11:55:26.166379 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:26.166364    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zc7qw"
Sep 11 11:55:26.169001 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:26.168977    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 11 11:55:26.169137 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:26.169007    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 11 11:55:26.169137 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:26.169018    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 11 11:55:26.169137 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:26.169121    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-fjhvz\""
Sep 11 11:55:26.169291 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:26.169275    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 11 11:55:26.182022 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:26.181999    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-zc7qw"]
Sep 11 11:55:26.231362 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:26.231323    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/93e10029-98cd-4ead-b26d-ed657714e363-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-zc7qw\" (UID: \"93e10029-98cd-4ead-b26d-ed657714e363\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zc7qw"
Sep 11 11:55:26.231362 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:26.231360    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/93e10029-98cd-4ead-b26d-ed657714e363-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-zc7qw\" (UID: \"93e10029-98cd-4ead-b26d-ed657714e363\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zc7qw"
Sep 11 11:55:26.231594 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:26.231456    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/93e10029-98cd-4ead-b26d-ed657714e363-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-zc7qw\" (UID: \"93e10029-98cd-4ead-b26d-ed657714e363\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zc7qw"
Sep 11 11:55:26.231594 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:26.231498    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/93e10029-98cd-4ead-b26d-ed657714e363-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-zc7qw\" (UID: \"93e10029-98cd-4ead-b26d-ed657714e363\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zc7qw"
Sep 11 11:55:26.231594 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:26.231534    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93e10029-98cd-4ead-b26d-ed657714e363-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-zc7qw\" (UID: \"93e10029-98cd-4ead-b26d-ed657714e363\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zc7qw"
Sep 11 11:55:26.231594 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:26.231580    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/93e10029-98cd-4ead-b26d-ed657714e363-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-zc7qw\" (UID: \"93e10029-98cd-4ead-b26d-ed657714e363\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zc7qw"
Sep 11 11:55:26.231742 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:26.231619    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jd5gq\" (UniqueName: \"kubernetes.io/projected/93e10029-98cd-4ead-b26d-ed657714e363-kube-api-access-jd5gq\") pod \"istiod-openshift-gateway-5677ccb49b-zc7qw\" (UID: \"93e10029-98cd-4ead-b26d-ed657714e363\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zc7qw"
Sep 11 11:55:26.332809 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:26.332777    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/93e10029-98cd-4ead-b26d-ed657714e363-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-zc7qw\" (UID: \"93e10029-98cd-4ead-b26d-ed657714e363\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zc7qw"
Sep 11 11:55:26.332809 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:26.332814    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/93e10029-98cd-4ead-b26d-ed657714e363-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-zc7qw\" (UID: \"93e10029-98cd-4ead-b26d-ed657714e363\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zc7qw"
Sep 11 11:55:26.332990 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:26.332843    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/93e10029-98cd-4ead-b26d-ed657714e363-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-zc7qw\" (UID: \"93e10029-98cd-4ead-b26d-ed657714e363\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zc7qw"
Sep 11 11:55:26.333051 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:26.333035    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/93e10029-98cd-4ead-b26d-ed657714e363-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-zc7qw\" (UID: \"93e10029-98cd-4ead-b26d-ed657714e363\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zc7qw"
Sep 11 11:55:26.333098 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:26.333086    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93e10029-98cd-4ead-b26d-ed657714e363-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-zc7qw\" (UID: \"93e10029-98cd-4ead-b26d-ed657714e363\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zc7qw"
Sep 11 11:55:26.333244 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:26.333219    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/93e10029-98cd-4ead-b26d-ed657714e363-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-zc7qw\" (UID: \"93e10029-98cd-4ead-b26d-ed657714e363\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zc7qw"
Sep 11 11:55:26.333323 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:26.333305    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jd5gq\" (UniqueName: \"kubernetes.io/projected/93e10029-98cd-4ead-b26d-ed657714e363-kube-api-access-jd5gq\") pod \"istiod-openshift-gateway-5677ccb49b-zc7qw\" (UID: \"93e10029-98cd-4ead-b26d-ed657714e363\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zc7qw"
Sep 11 11:55:26.334029 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:26.334001    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/93e10029-98cd-4ead-b26d-ed657714e363-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-zc7qw\" (UID: \"93e10029-98cd-4ead-b26d-ed657714e363\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zc7qw"
Sep 11 11:55:26.335271 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:26.335242    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/93e10029-98cd-4ead-b26d-ed657714e363-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-zc7qw\" (UID: \"93e10029-98cd-4ead-b26d-ed657714e363\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zc7qw"
Sep 11 11:55:26.335382 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:26.335359    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/93e10029-98cd-4ead-b26d-ed657714e363-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-zc7qw\" (UID: \"93e10029-98cd-4ead-b26d-ed657714e363\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zc7qw"
Sep 11 11:55:26.335461 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:26.335367    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/93e10029-98cd-4ead-b26d-ed657714e363-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-zc7qw\" (UID: \"93e10029-98cd-4ead-b26d-ed657714e363\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zc7qw"
Sep 11 11:55:26.335853 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:26.335833    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/93e10029-98cd-4ead-b26d-ed657714e363-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-zc7qw\" (UID: \"93e10029-98cd-4ead-b26d-ed657714e363\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zc7qw"
Sep 11 11:55:26.345972 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:26.345948    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93e10029-98cd-4ead-b26d-ed657714e363-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-zc7qw\" (UID: \"93e10029-98cd-4ead-b26d-ed657714e363\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zc7qw"
Sep 11 11:55:26.347375 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:26.347355    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jd5gq\" (UniqueName: \"kubernetes.io/projected/93e10029-98cd-4ead-b26d-ed657714e363-kube-api-access-jd5gq\") pod \"istiod-openshift-gateway-5677ccb49b-zc7qw\" (UID: \"93e10029-98cd-4ead-b26d-ed657714e363\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zc7qw"
Sep 11 11:55:26.476089 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:26.476006    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zc7qw"
Sep 11 11:55:26.757046 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:26.757022    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-zc7qw"]
Sep 11 11:55:26.759657 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:55:26.759627    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93e10029_98cd_4ead_b26d_ed657714e363.slice/crio-f08b0e458206a2002ed7026fb4d31f9cb1d2e07c117e7bf67e9cd46c10dc549b WatchSource:0}: Error finding container f08b0e458206a2002ed7026fb4d31f9cb1d2e07c117e7bf67e9cd46c10dc549b: Status 404 returned error can't find the container with id f08b0e458206a2002ed7026fb4d31f9cb1d2e07c117e7bf67e9cd46c10dc549b
Sep 11 11:55:27.384177 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:27.384131    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zc7qw" event={"ID":"93e10029-98cd-4ead-b26d-ed657714e363","Type":"ContainerStarted","Data":"f08b0e458206a2002ed7026fb4d31f9cb1d2e07c117e7bf67e9cd46c10dc549b"}
Sep 11 11:55:29.086915 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:29.086868    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 11:55:29.087205 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:29.086970    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 11:55:29.395339 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:29.395305    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zc7qw" event={"ID":"93e10029-98cd-4ead-b26d-ed657714e363","Type":"ContainerStarted","Data":"3c9a94857ca8a8a2b2f344ddb147ad24e1d0cd5245b5ed138da8e33f9976eed6"}
Sep 11 11:55:29.395566 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:29.395439    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zc7qw"
Sep 11 11:55:29.440183 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:29.440125    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zc7qw" podStartSLOduration=1.115321944 podStartE2EDuration="3.440108746s" totalImagesPullingTime="2.324786802s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:55:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:55:29.434478795 +0000 UTC m=+433.331920541" watchObservedRunningTime="2026-09-11 11:55:29.440108746 +0000 UTC m=+433.337550489"
Sep 11 11:55:29.687220 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:29.687133    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:55:29.828071 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:29.828040    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-t4h64"]
Sep 11 11:55:29.832038 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:29.832020    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-t4h64"
Sep 11 11:55:29.835712 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:29.835684    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-lzzxd\""
Sep 11 11:55:29.835879 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:29.835859    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 11 11:55:29.845484 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:29.845454    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-vkf9d"]
Sep 11 11:55:29.849187 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:29.849169    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vkf9d"
Sep 11 11:55:29.851793 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:29.851772    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 11 11:55:29.851937 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:29.851920    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-x6672\""
Sep 11 11:55:29.862749 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:29.862722    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-t4h64"]
Sep 11 11:55:29.868918 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:29.868831    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-vkf9d"]
Sep 11 11:55:29.877885 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:29.877741    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:55:29.969597 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:29.969506    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/946b34d0-c922-458b-8b7e-b6de54bbcd05-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-vkf9d\" (UID: \"946b34d0-c922-458b-8b7e-b6de54bbcd05\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vkf9d"
Sep 11 11:55:29.969597 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:29.969549    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcnqh\" (UniqueName: \"kubernetes.io/projected/946b34d0-c922-458b-8b7e-b6de54bbcd05-kube-api-access-jcnqh\") pod \"llmisvc-controller-manager-777c6bd7d5-vkf9d\" (UID: \"946b34d0-c922-458b-8b7e-b6de54bbcd05\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vkf9d"
Sep 11 11:55:29.969793 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:29.969625    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tv82x\" (UniqueName: \"kubernetes.io/projected/1d089f0b-5e09-440a-a8e5-cbe5bf59b1f2-kube-api-access-tv82x\") pod \"kserve-controller-manager-d6f4b65b8-t4h64\" (UID: \"1d089f0b-5e09-440a-a8e5-cbe5bf59b1f2\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-t4h64"
Sep 11 11:55:29.969793 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:29.969669    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1d089f0b-5e09-440a-a8e5-cbe5bf59b1f2-cert\") pod \"kserve-controller-manager-d6f4b65b8-t4h64\" (UID: \"1d089f0b-5e09-440a-a8e5-cbe5bf59b1f2\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-t4h64"
Sep 11 11:55:30.011282 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:30.011244    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:55:30.070884 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:30.070841    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/946b34d0-c922-458b-8b7e-b6de54bbcd05-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-vkf9d\" (UID: \"946b34d0-c922-458b-8b7e-b6de54bbcd05\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vkf9d"
Sep 11 11:55:30.071080 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:30.070895    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcnqh\" (UniqueName: \"kubernetes.io/projected/946b34d0-c922-458b-8b7e-b6de54bbcd05-kube-api-access-jcnqh\") pod \"llmisvc-controller-manager-777c6bd7d5-vkf9d\" (UID: \"946b34d0-c922-458b-8b7e-b6de54bbcd05\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vkf9d"
Sep 11 11:55:30.071080 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:30.070955    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tv82x\" (UniqueName: \"kubernetes.io/projected/1d089f0b-5e09-440a-a8e5-cbe5bf59b1f2-kube-api-access-tv82x\") pod \"kserve-controller-manager-d6f4b65b8-t4h64\" (UID: \"1d089f0b-5e09-440a-a8e5-cbe5bf59b1f2\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-t4h64"
Sep 11 11:55:30.071080 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:30.071004    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1d089f0b-5e09-440a-a8e5-cbe5bf59b1f2-cert\") pod \"kserve-controller-manager-d6f4b65b8-t4h64\" (UID: \"1d089f0b-5e09-440a-a8e5-cbe5bf59b1f2\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-t4h64"
Sep 11 11:55:30.073426 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:30.073372    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/946b34d0-c922-458b-8b7e-b6de54bbcd05-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-vkf9d\" (UID: \"946b34d0-c922-458b-8b7e-b6de54bbcd05\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vkf9d"
Sep 11 11:55:30.073631 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:30.073610    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1d089f0b-5e09-440a-a8e5-cbe5bf59b1f2-cert\") pod \"kserve-controller-manager-d6f4b65b8-t4h64\" (UID: \"1d089f0b-5e09-440a-a8e5-cbe5bf59b1f2\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-t4h64"
Sep 11 11:55:30.100266 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:30.100233    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcnqh\" (UniqueName: \"kubernetes.io/projected/946b34d0-c922-458b-8b7e-b6de54bbcd05-kube-api-access-jcnqh\") pod \"llmisvc-controller-manager-777c6bd7d5-vkf9d\" (UID: \"946b34d0-c922-458b-8b7e-b6de54bbcd05\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vkf9d"
Sep 11 11:55:30.100604 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:30.100302    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tv82x\" (UniqueName: \"kubernetes.io/projected/1d089f0b-5e09-440a-a8e5-cbe5bf59b1f2-kube-api-access-tv82x\") pod \"kserve-controller-manager-d6f4b65b8-t4h64\" (UID: \"1d089f0b-5e09-440a-a8e5-cbe5bf59b1f2\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-t4h64"
Sep 11 11:55:30.142220 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:30.142180    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-t4h64"
Sep 11 11:55:30.159701 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:30.159670    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vkf9d"
Sep 11 11:55:30.403037 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:30.402999    2579 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-zc7qw container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 11 11:55:30.403180 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:30.403081    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zc7qw" podUID="93e10029-98cd-4ead-b26d-ed657714e363" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 11:55:30.473930 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:30.473901    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-t4h64"]
Sep 11 11:55:30.475527 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:55:30.475493    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d089f0b_5e09_440a_a8e5_cbe5bf59b1f2.slice/crio-d613786593ae178f60641d72899785335d250fbf580e341a2dd72603f90f2137 WatchSource:0}: Error finding container d613786593ae178f60641d72899785335d250fbf580e341a2dd72603f90f2137: Status 404 returned error can't find the container with id d613786593ae178f60641d72899785335d250fbf580e341a2dd72603f90f2137
Sep 11 11:55:30.526635 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:30.526596    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-vkf9d"]
Sep 11 11:55:30.533252 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:55:30.533216    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod946b34d0_c922_458b_8b7e_b6de54bbcd05.slice/crio-738832fb2fcf0ce502f3edd6ed4d967253e70bfa243966768abd0113fa3c801b WatchSource:0}: Error finding container 738832fb2fcf0ce502f3edd6ed4d967253e70bfa243966768abd0113fa3c801b: Status 404 returned error can't find the container with id 738832fb2fcf0ce502f3edd6ed4d967253e70bfa243966768abd0113fa3c801b
Sep 11 11:55:31.109646 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:31.109616    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:55:31.328352 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:31.328312    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:55:31.410347 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:31.410271    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vkf9d" event={"ID":"946b34d0-c922-458b-8b7e-b6de54bbcd05","Type":"ContainerStarted","Data":"738832fb2fcf0ce502f3edd6ed4d967253e70bfa243966768abd0113fa3c801b"}
Sep 11 11:55:31.411857 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:31.411823    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-t4h64" event={"ID":"1d089f0b-5e09-440a-a8e5-cbe5bf59b1f2","Type":"ContainerStarted","Data":"d613786593ae178f60641d72899785335d250fbf580e341a2dd72603f90f2137"}
Sep 11 11:55:33.402229 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:33.402193    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zc7qw"
Sep 11 11:55:34.368071 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:34.368040    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-mvbmv"
Sep 11 11:55:34.369844 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:34.369821    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-4zb6j"
Sep 11 11:55:34.430374 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:34.430337    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vkf9d" event={"ID":"946b34d0-c922-458b-8b7e-b6de54bbcd05","Type":"ContainerStarted","Data":"79cebde5111c0d3ac1ca1eb5eb537541f3219b5b69d7ea06c59d0ac9c6239965"}
Sep 11 11:55:34.430833 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:34.430463    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vkf9d"
Sep 11 11:55:34.432298 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:34.432276    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-t4h64" event={"ID":"1d089f0b-5e09-440a-a8e5-cbe5bf59b1f2","Type":"ContainerStarted","Data":"41db9bdb6a87ed0857b701b4be686475b0c4f09a9406b3039246ed7b3469d26a"}
Sep 11 11:55:34.432456 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:34.432326    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-t4h64"
Sep 11 11:55:34.450238 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:34.450185    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vkf9d" podStartSLOduration=2.462082387 podStartE2EDuration="5.450169722s" totalImagesPullingTime="2.988087335s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:55:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:55:34.450076896 +0000 UTC m=+438.347518662" watchObservedRunningTime="2026-09-11 11:55:34.450169722 +0000 UTC m=+438.347611480"
Sep 11 11:55:34.468833 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:55:34.468783    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-t4h64" podStartSLOduration=2.479713893 podStartE2EDuration="5.468766966s" totalImagesPullingTime="2.989053073s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:55:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:55:34.467375974 +0000 UTC m=+438.364817723" watchObservedRunningTime="2026-09-11 11:55:34.468766966 +0000 UTC m=+438.366208711"
Sep 11 11:56:05.438778 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:56:05.438748    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vkf9d"
Sep 11 11:56:05.442222 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:56:05.442199    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-t4h64"
Sep 11 11:56:19.882566 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:56:19.882532    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:19.978881 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:56:19.978846    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:29.676942 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:56:29.676904    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:29.769439 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:56:29.769401    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:29.869856 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:56:29.869816    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:30.393267 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:56:30.393225    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:30.468026 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:56:30.467987    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:36.663738 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:56:36.663699    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:37.308980 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:56:37.308950    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:39.685879 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:56:39.685843    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:39.812901 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:56:39.812865    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-wwjmx"]
Sep 11 11:56:39.815147 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:56:39.815129    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-wwjmx"
Sep 11 11:56:39.819338 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:56:39.819319    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 11 11:56:39.819505 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:56:39.819374    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-tlq6s\""
Sep 11 11:56:39.820171 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:56:39.820153    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 11 11:56:39.839051 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:56:39.839025    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-wwjmx"]
Sep 11 11:56:39.901790 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:56:39.901752    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvhxc\" (UniqueName: \"kubernetes.io/projected/14b00de4-0665-4db5-90c6-5af92818fcc9-kube-api-access-rvhxc\") pod \"authorino-operator-76b478bd78-wwjmx\" (UID: \"14b00de4-0665-4db5-90c6-5af92818fcc9\") " pod="kuadrant-system/authorino-operator-76b478bd78-wwjmx"
Sep 11 11:56:40.002730 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:56:40.002632    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvhxc\" (UniqueName: \"kubernetes.io/projected/14b00de4-0665-4db5-90c6-5af92818fcc9-kube-api-access-rvhxc\") pod \"authorino-operator-76b478bd78-wwjmx\" (UID: \"14b00de4-0665-4db5-90c6-5af92818fcc9\") " pod="kuadrant-system/authorino-operator-76b478bd78-wwjmx"
Sep 11 11:56:40.016190 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:56:40.016156    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvhxc\" (UniqueName: \"kubernetes.io/projected/14b00de4-0665-4db5-90c6-5af92818fcc9-kube-api-access-rvhxc\") pod \"authorino-operator-76b478bd78-wwjmx\" (UID: \"14b00de4-0665-4db5-90c6-5af92818fcc9\") " pod="kuadrant-system/authorino-operator-76b478bd78-wwjmx"
Sep 11 11:56:40.126204 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:56:40.126166    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-wwjmx"
Sep 11 11:56:40.194724 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:56:40.194691    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:40.299738 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:56:40.299653    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:40.417762 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:56:40.417611    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-wwjmx"]
Sep 11 11:56:40.417857 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:56:40.417822    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:40.420637 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:56:40.420609    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14b00de4_0665_4db5_90c6_5af92818fcc9.slice/crio-b7c17b28801540e966811e92743d22588fd26705433d6162b1192e78c38d9678 WatchSource:0}: Error finding container b7c17b28801540e966811e92743d22588fd26705433d6162b1192e78c38d9678: Status 404 returned error can't find the container with id b7c17b28801540e966811e92743d22588fd26705433d6162b1192e78c38d9678
Sep 11 11:56:40.610508 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:56:40.610469    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:40.707144 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:56:40.707106    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:40.731715 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:56:40.731683    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-wwjmx" event={"ID":"14b00de4-0665-4db5-90c6-5af92818fcc9","Type":"ContainerStarted","Data":"b7c17b28801540e966811e92743d22588fd26705433d6162b1192e78c38d9678"}
Sep 11 11:56:40.879521 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:56:40.879428    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:41.056833 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:56:41.056788    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:41.449953 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:56:41.449917    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:42.401726 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:56:42.401684    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:42.537953 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:56:42.537911    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:43.750368 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:56:43.750332    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-wwjmx" event={"ID":"14b00de4-0665-4db5-90c6-5af92818fcc9","Type":"ContainerStarted","Data":"435f276c4c7ee87cae09d493307c676f2bc16aec115670924b35de75aa08205d"}
Sep 11 11:56:43.750768 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:56:43.750550    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-wwjmx"
Sep 11 11:56:43.785671 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:56:43.785613    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-76b478bd78-wwjmx" podStartSLOduration=2.025741269 podStartE2EDuration="4.785595817s" totalImagesPullingTime="2.759854548s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:56:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:56:43.783478325 +0000 UTC m=+507.680920064" watchObservedRunningTime="2026-09-11 11:56:43.785595817 +0000 UTC m=+507.683037561"
Sep 11 11:56:44.924622 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:56:44.924589    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:46.946319 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:56:46.946282    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:47.492741 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:56:47.492707    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:48.629061 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:56:48.629024    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:48.668463 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:56:48.668430    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-9n85g"]
Sep 11 11:56:48.670931 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:56:48.670911    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9n85g"
Sep 11 11:56:48.674657 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:56:48.674629    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-s5mnj\""
Sep 11 11:56:48.694959 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:56:48.694929    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-9n85g"]
Sep 11 11:56:48.774220 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:56:48.774183    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrtvq\" (UniqueName: \"kubernetes.io/projected/a34baa01-7f23-4d57-b4d3-9bc5ff6f5350-kube-api-access-rrtvq\") pod \"limitador-operator-controller-manager-74495c69f8-9n85g\" (UID: \"a34baa01-7f23-4d57-b4d3-9bc5ff6f5350\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9n85g"
Sep 11 11:56:48.874769 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:56:48.874731    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrtvq\" (UniqueName: \"kubernetes.io/projected/a34baa01-7f23-4d57-b4d3-9bc5ff6f5350-kube-api-access-rrtvq\") pod \"limitador-operator-controller-manager-74495c69f8-9n85g\" (UID: \"a34baa01-7f23-4d57-b4d3-9bc5ff6f5350\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9n85g"
Sep 11 11:56:48.886445 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:56:48.886347    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrtvq\" (UniqueName: \"kubernetes.io/projected/a34baa01-7f23-4d57-b4d3-9bc5ff6f5350-kube-api-access-rrtvq\") pod \"limitador-operator-controller-manager-74495c69f8-9n85g\" (UID: \"a34baa01-7f23-4d57-b4d3-9bc5ff6f5350\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9n85g"
Sep 11 11:56:48.981038 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:56:48.981010    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9n85g"
Sep 11 11:56:49.265788 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:56:49.265760    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-9n85g"]
Sep 11 11:56:49.267908 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:56:49.267881    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda34baa01_7f23_4d57_b4d3_9bc5ff6f5350.slice/crio-5015b861d728f887ad83c40608854c2262c3e02731b6e0fad12f9d6af485274b WatchSource:0}: Error finding container 5015b861d728f887ad83c40608854c2262c3e02731b6e0fad12f9d6af485274b: Status 404 returned error can't find the container with id 5015b861d728f887ad83c40608854c2262c3e02731b6e0fad12f9d6af485274b
Sep 11 11:56:49.780021 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:56:49.779986    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9n85g" event={"ID":"a34baa01-7f23-4d57-b4d3-9bc5ff6f5350","Type":"ContainerStarted","Data":"5015b861d728f887ad83c40608854c2262c3e02731b6e0fad12f9d6af485274b"}
Sep 11 11:56:50.209596 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:56:50.209565    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:50.304919 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:56:50.304879    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:50.534980 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:56:50.534882    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:50.631518 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:56:50.631486    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:51.792541 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:56:51.792495    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9n85g" event={"ID":"a34baa01-7f23-4d57-b4d3-9bc5ff6f5350","Type":"ContainerStarted","Data":"203526fff67d9804e2d543f43379cd6fcb8776c05fbd17e382bfb6244a86dfda"}
Sep 11 11:56:51.793009 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:56:51.792592    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9n85g"
Sep 11 11:56:51.811255 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:56:51.811204    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9n85g" podStartSLOduration=2.064360788 podStartE2EDuration="3.811189786s" totalImagesPullingTime="1.746828998s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:56:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:56:51.81020127 +0000 UTC m=+515.707643016" watchObservedRunningTime="2026-09-11 11:56:51.811189786 +0000 UTC m=+515.708631530"
Sep 11 11:56:52.044903 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:56:52.044816    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:52.120867 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:56:52.120835    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:54.756860 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:56:54.756828    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-wwjmx"
Sep 11 11:56:54.832793 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:56:54.832759    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:54.924078 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:56:54.924034    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:55.023250 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:56:55.023159    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:55.144633 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:56:55.144601    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:55.243502 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:56:55.243468    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:57:01.174711 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:01.174657    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:57:01.319234 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:01.319199    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:57:02.075049 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:02.075012    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:57:02.276985 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:02.276950    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:57:02.800535 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:02.800507    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9n85g"
Sep 11 11:57:03.301482 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:03.301437    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-9n85g"]
Sep 11 11:57:03.301927 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:03.301772    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9n85g" podUID="a34baa01-7f23-4d57-b4d3-9bc5ff6f5350" containerName="manager" containerID="cri-o://203526fff67d9804e2d543f43379cd6fcb8776c05fbd17e382bfb6244a86dfda" gracePeriod=2
Sep 11 11:57:03.304020 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:57:03.303986    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-9n85g\" is forbidden: User \"system:node:ip-10-0-142-145.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-145.ec2.internal' and this object" podUID="a34baa01-7f23-4d57-b4d3-9bc5ff6f5350" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9n85g"
Sep 11 11:57:03.304146 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:03.304126    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-9n85g"]
Sep 11 11:57:03.353678 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:03.353637    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-stlxq"]
Sep 11 11:57:03.354268 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:03.354247    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a34baa01-7f23-4d57-b4d3-9bc5ff6f5350" containerName="manager"
Sep 11 11:57:03.354354 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:03.354273    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a34baa01-7f23-4d57-b4d3-9bc5ff6f5350" containerName="manager"
Sep 11 11:57:03.354426 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:03.354374    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a34baa01-7f23-4d57-b4d3-9bc5ff6f5350" containerName="manager"
Sep 11 11:57:03.357744 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:03.357719    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-stlxq"
Sep 11 11:57:03.360662 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:57:03.360634    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-9n85g\" is forbidden: User \"system:node:ip-10-0-142-145.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-145.ec2.internal' and this object" podUID="a34baa01-7f23-4d57-b4d3-9bc5ff6f5350" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9n85g"
Sep 11 11:57:03.380160 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:03.380132    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-stlxq"]
Sep 11 11:57:03.406160 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:03.406129    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gl8ts\" (UniqueName: \"kubernetes.io/projected/fe3781b9-2a1e-4f3f-93ea-29b2a27e85e6-kube-api-access-gl8ts\") pod \"limitador-operator-controller-manager-74495c69f8-stlxq\" (UID: \"fe3781b9-2a1e-4f3f-93ea-29b2a27e85e6\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-stlxq"
Sep 11 11:57:03.506988 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:03.506949    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gl8ts\" (UniqueName: \"kubernetes.io/projected/fe3781b9-2a1e-4f3f-93ea-29b2a27e85e6-kube-api-access-gl8ts\") pod \"limitador-operator-controller-manager-74495c69f8-stlxq\" (UID: \"fe3781b9-2a1e-4f3f-93ea-29b2a27e85e6\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-stlxq"
Sep 11 11:57:03.517306 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:03.517271    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gl8ts\" (UniqueName: \"kubernetes.io/projected/fe3781b9-2a1e-4f3f-93ea-29b2a27e85e6-kube-api-access-gl8ts\") pod \"limitador-operator-controller-manager-74495c69f8-stlxq\" (UID: \"fe3781b9-2a1e-4f3f-93ea-29b2a27e85e6\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-stlxq"
Sep 11 11:57:03.544913 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:03.544887    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9n85g"
Sep 11 11:57:03.547743 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:57:03.547715    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-9n85g\" is forbidden: User \"system:node:ip-10-0-142-145.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-145.ec2.internal' and this object" podUID="a34baa01-7f23-4d57-b4d3-9bc5ff6f5350" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9n85g"
Sep 11 11:57:03.608360 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:03.608326    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rrtvq\" (UniqueName: \"kubernetes.io/projected/a34baa01-7f23-4d57-b4d3-9bc5ff6f5350-kube-api-access-rrtvq\") pod \"a34baa01-7f23-4d57-b4d3-9bc5ff6f5350\" (UID: \"a34baa01-7f23-4d57-b4d3-9bc5ff6f5350\") "
Sep 11 11:57:03.610571 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:03.610541    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a34baa01-7f23-4d57-b4d3-9bc5ff6f5350-kube-api-access-rrtvq" (OuterVolumeSpecName: "kube-api-access-rrtvq") pod "a34baa01-7f23-4d57-b4d3-9bc5ff6f5350" (UID: "a34baa01-7f23-4d57-b4d3-9bc5ff6f5350"). InnerVolumeSpecName "kube-api-access-rrtvq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 11:57:03.688922 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:03.688886    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-stlxq"
Sep 11 11:57:03.709931 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:03.709900    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rrtvq\" (UniqueName: \"kubernetes.io/projected/a34baa01-7f23-4d57-b4d3-9bc5ff6f5350-kube-api-access-rrtvq\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 11:57:03.850944 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:03.850910    2579 generic.go:356] "Generic (PLEG): container finished" podID="a34baa01-7f23-4d57-b4d3-9bc5ff6f5350" containerID="203526fff67d9804e2d543f43379cd6fcb8776c05fbd17e382bfb6244a86dfda" exitCode=0
Sep 11 11:57:03.851153 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:03.850960    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9n85g"
Sep 11 11:57:03.851153 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:03.851011    2579 scope.go:117] "RemoveContainer" containerID="203526fff67d9804e2d543f43379cd6fcb8776c05fbd17e382bfb6244a86dfda"
Sep 11 11:57:03.853968 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:57:03.853940    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-9n85g\" is forbidden: User \"system:node:ip-10-0-142-145.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-145.ec2.internal' and this object" podUID="a34baa01-7f23-4d57-b4d3-9bc5ff6f5350" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9n85g"
Sep 11 11:57:03.860154 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:03.860133    2579 scope.go:117] "RemoveContainer" containerID="203526fff67d9804e2d543f43379cd6fcb8776c05fbd17e382bfb6244a86dfda"
Sep 11 11:57:03.860477 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:57:03.860456    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"203526fff67d9804e2d543f43379cd6fcb8776c05fbd17e382bfb6244a86dfda\": container with ID starting with 203526fff67d9804e2d543f43379cd6fcb8776c05fbd17e382bfb6244a86dfda not found: ID does not exist" containerID="203526fff67d9804e2d543f43379cd6fcb8776c05fbd17e382bfb6244a86dfda"
Sep 11 11:57:03.860559 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:03.860486    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"203526fff67d9804e2d543f43379cd6fcb8776c05fbd17e382bfb6244a86dfda"} err="failed to get container status \"203526fff67d9804e2d543f43379cd6fcb8776c05fbd17e382bfb6244a86dfda\": rpc error: code = NotFound desc = could not find container \"203526fff67d9804e2d543f43379cd6fcb8776c05fbd17e382bfb6244a86dfda\": container with ID starting with 203526fff67d9804e2d543f43379cd6fcb8776c05fbd17e382bfb6244a86dfda not found: ID does not exist"
Sep 11 11:57:03.862800 ip-10-0-142-145 kubenswrapper[2579]: E0911 11:57:03.862777    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-9n85g\" is forbidden: User \"system:node:ip-10-0-142-145.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-145.ec2.internal' and this object" podUID="a34baa01-7f23-4d57-b4d3-9bc5ff6f5350" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9n85g"
Sep 11 11:57:03.979637 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:03.979597    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-stlxq"]
Sep 11 11:57:03.983678 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:57:03.983651    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe3781b9_2a1e_4f3f_93ea_29b2a27e85e6.slice/crio-d9669eb2b3a3aff75b646a77e852b89d1d7b3bbd107ba34763fe2815e4e5a8cf WatchSource:0}: Error finding container d9669eb2b3a3aff75b646a77e852b89d1d7b3bbd107ba34763fe2815e4e5a8cf: Status 404 returned error can't find the container with id d9669eb2b3a3aff75b646a77e852b89d1d7b3bbd107ba34763fe2815e4e5a8cf
Sep 11 11:57:04.575801 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:04.575760    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a34baa01-7f23-4d57-b4d3-9bc5ff6f5350" path="/var/lib/kubelet/pods/a34baa01-7f23-4d57-b4d3-9bc5ff6f5350/volumes"
Sep 11 11:57:04.858003 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:04.857969    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-stlxq" event={"ID":"fe3781b9-2a1e-4f3f-93ea-29b2a27e85e6","Type":"ContainerStarted","Data":"65222a0ef71268c38a6d609e04b74a80167967deac7f6670d3371c57afdb32ce"}
Sep 11 11:57:04.858171 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:04.858025    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-stlxq" event={"ID":"fe3781b9-2a1e-4f3f-93ea-29b2a27e85e6","Type":"ContainerStarted","Data":"d9669eb2b3a3aff75b646a77e852b89d1d7b3bbd107ba34763fe2815e4e5a8cf"}
Sep 11 11:57:04.858171 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:04.858105    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-stlxq"
Sep 11 11:57:04.879976 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:04.879933    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-stlxq" podStartSLOduration=1.879918424 podStartE2EDuration="1.879918424s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:57:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:57:04.879299998 +0000 UTC m=+528.776741764" watchObservedRunningTime="2026-09-11 11:57:04.879918424 +0000 UTC m=+528.777360169"
Sep 11 11:57:05.891366 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:05.891328    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:57:06.026947 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:06.026917    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:57:09.291208 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:09.291123    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:57:09.617916 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:09.617880    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:57:09.901433 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:09.901323    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:57:10.131574 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:10.131542    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:57:14.884711 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:14.884669    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:57:15.221233 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:15.221145    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:57:15.315819 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:15.315785    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:57:15.420793 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:15.420758    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:57:15.680880 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:15.680844    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:57:15.784764 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:15.784727    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:57:15.865627 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:15.865597    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-stlxq"
Sep 11 11:57:21.079589 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:21.079553    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:57:21.176311 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:21.176276    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:57:25.808319 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:25.808284    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:57:25.908193 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:25.908144    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:57:33.199068 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:33.199018    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:57:33.324950 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:33.324912    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:57:34.956685 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:34.956642    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:57:35.078989 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:35.078957    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:57:35.184930 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:35.184892    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:57:35.494883 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:35.494847    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 11:57:35.499685 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:35.499669    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-znhkx"
Sep 11 11:57:35.502773 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:35.502753    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-h7kcd\""
Sep 11 11:57:35.502980 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:35.502963    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 11 11:57:35.523995 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:35.523966    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 11:57:35.596687 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:35.596649    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brj5v\" (UniqueName: \"kubernetes.io/projected/a2f763ac-c4a1-4ca5-866a-a10c3c64a893-kube-api-access-brj5v\") pod \"limitador-limitador-84994f7ffc-znhkx\" (UID: \"a2f763ac-c4a1-4ca5-866a-a10c3c64a893\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-znhkx"
Sep 11 11:57:35.596864 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:35.596704    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a2f763ac-c4a1-4ca5-866a-a10c3c64a893-config-file\") pod \"limitador-limitador-84994f7ffc-znhkx\" (UID: \"a2f763ac-c4a1-4ca5-866a-a10c3c64a893\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-znhkx"
Sep 11 11:57:35.628556 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:35.628522    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 11:57:35.697960 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:35.697918    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brj5v\" (UniqueName: \"kubernetes.io/projected/a2f763ac-c4a1-4ca5-866a-a10c3c64a893-kube-api-access-brj5v\") pod \"limitador-limitador-84994f7ffc-znhkx\" (UID: \"a2f763ac-c4a1-4ca5-866a-a10c3c64a893\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-znhkx"
Sep 11 11:57:35.698148 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:35.697980    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a2f763ac-c4a1-4ca5-866a-a10c3c64a893-config-file\") pod \"limitador-limitador-84994f7ffc-znhkx\" (UID: \"a2f763ac-c4a1-4ca5-866a-a10c3c64a893\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-znhkx"
Sep 11 11:57:35.698688 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:35.698655    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a2f763ac-c4a1-4ca5-866a-a10c3c64a893-config-file\") pod \"limitador-limitador-84994f7ffc-znhkx\" (UID: \"a2f763ac-c4a1-4ca5-866a-a10c3c64a893\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-znhkx"
Sep 11 11:57:35.728014 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:35.727977    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brj5v\" (UniqueName: \"kubernetes.io/projected/a2f763ac-c4a1-4ca5-866a-a10c3c64a893-kube-api-access-brj5v\") pod \"limitador-limitador-84994f7ffc-znhkx\" (UID: \"a2f763ac-c4a1-4ca5-866a-a10c3c64a893\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-znhkx"
Sep 11 11:57:35.809478 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:35.809371    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-znhkx"
Sep 11 11:57:36.013709 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:36.013679    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:57:36.094966 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:36.094897    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 11:57:36.097602 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:57:36.097575    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2f763ac_c4a1_4ca5_866a_a10c3c64a893.slice/crio-f0544599be33c18e2fb93afefedafb64b63ff695b5f1a3c37d3e49e6516f70ca WatchSource:0}: Error finding container f0544599be33c18e2fb93afefedafb64b63ff695b5f1a3c37d3e49e6516f70ca: Status 404 returned error can't find the container with id f0544599be33c18e2fb93afefedafb64b63ff695b5f1a3c37d3e49e6516f70ca
Sep 11 11:57:36.132119 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:36.132085    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:57:37.022488 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:37.022443    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-znhkx" event={"ID":"a2f763ac-c4a1-4ca5-866a-a10c3c64a893","Type":"ContainerStarted","Data":"f0544599be33c18e2fb93afefedafb64b63ff695b5f1a3c37d3e49e6516f70ca"}
Sep 11 11:57:38.028347 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:38.028307    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-znhkx" event={"ID":"a2f763ac-c4a1-4ca5-866a-a10c3c64a893","Type":"ContainerStarted","Data":"78ba1fa7cb360cd5ec6934ef58afaa038c3861ad1e41f420f4205d615067fbca"}
Sep 11 11:57:38.028802 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:38.028438    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-znhkx"
Sep 11 11:57:38.050212 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:38.050161    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-84994f7ffc-znhkx" podStartSLOduration=1.935679121 podStartE2EDuration="3.050146886s" totalImagesPullingTime="1.114467765s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:57:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:57:38.04871033 +0000 UTC m=+561.946152076" watchObservedRunningTime="2026-09-11 11:57:38.050146886 +0000 UTC m=+561.947588630"
Sep 11 11:57:49.035337 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:49.035306    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-znhkx"
Sep 11 11:57:51.029120 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:51.029084    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:57:51.128847 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:51.128811    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:57:51.500893 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:57:51.500854    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:04.119620 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:58:04.119584    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:04.988777 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:58:04.988742    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:10.699969 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:58:10.699928    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:10.784791 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:58:10.784752    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:10.854107 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:58:10.854034    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:10.937616 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:58:10.937581    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:13.826952 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:58:13.826913    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:13.915505 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:58:13.915471    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:16.478502 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:58:16.478473    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-ch96n_1f0f9fa5-cbd1-468b-ae81-82b48c101d37/console-operator/2.log"
Sep 11 11:58:16.480654 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:58:16.480629    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-ch96n_1f0f9fa5-cbd1-468b-ae81-82b48c101d37/console-operator/2.log"
Sep 11 11:58:20.970118 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:58:20.970086    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:47.459441 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:58:47.459406    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:47.557941 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:58:47.557906    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:49.074071 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:58:49.074031    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:49.186486 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:58:49.186450    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:49.272492 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:58:49.272456    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:49.369441 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:58:49.369409    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:49.757431 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:58:49.757289    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:49.924565 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:58:49.924531    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 11:58:49.925095 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:58:49.925064    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:58.636492 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:58:58.636450    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:59:00.292787 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:00.292754    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:59:00.450327 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:00.450297    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:59:04.666243 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:04.666207    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:59:04.742131 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:04.742095    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:59:09.780049 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:09.780014    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:59:10.366581 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:10.366536    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 11:59:14.240719 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:14.240682    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-gcbd8"]
Sep 11 11:59:14.244542 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:14.244517    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gcbd8"
Sep 11 11:59:14.248434 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:14.248354    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-fg5rd\""
Sep 11 11:59:14.248434 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:14.248355    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 11 11:59:14.248434 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:14.248438    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 11 11:59:14.248705 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:14.248355    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 11 11:59:14.256144 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:14.256118    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-gcbd8"]
Sep 11 11:59:14.279889 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:14.279852    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:59:14.292949 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:14.292918    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84f2fd48-b014-4d96-b247-44a77932b73c-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gcbd8\" (UID: \"84f2fd48-b014-4d96-b247-44a77932b73c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gcbd8"
Sep 11 11:59:14.293128 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:14.292983    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84f2fd48-b014-4d96-b247-44a77932b73c-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gcbd8\" (UID: \"84f2fd48-b014-4d96-b247-44a77932b73c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gcbd8"
Sep 11 11:59:14.293128 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:14.293008    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84f2fd48-b014-4d96-b247-44a77932b73c-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gcbd8\" (UID: \"84f2fd48-b014-4d96-b247-44a77932b73c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gcbd8"
Sep 11 11:59:14.293128 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:14.293075    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84f2fd48-b014-4d96-b247-44a77932b73c-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gcbd8\" (UID: \"84f2fd48-b014-4d96-b247-44a77932b73c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gcbd8"
Sep 11 11:59:14.293266 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:14.293136    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gfn8\" (UniqueName: \"kubernetes.io/projected/84f2fd48-b014-4d96-b247-44a77932b73c-kube-api-access-6gfn8\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gcbd8\" (UID: \"84f2fd48-b014-4d96-b247-44a77932b73c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gcbd8"
Sep 11 11:59:14.293266 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:14.293158    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84f2fd48-b014-4d96-b247-44a77932b73c-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gcbd8\" (UID: \"84f2fd48-b014-4d96-b247-44a77932b73c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gcbd8"
Sep 11 11:59:14.394341 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:14.394306    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84f2fd48-b014-4d96-b247-44a77932b73c-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gcbd8\" (UID: \"84f2fd48-b014-4d96-b247-44a77932b73c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gcbd8"
Sep 11 11:59:14.394341 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:14.394344    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84f2fd48-b014-4d96-b247-44a77932b73c-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gcbd8\" (UID: \"84f2fd48-b014-4d96-b247-44a77932b73c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gcbd8"
Sep 11 11:59:14.394596 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:14.394468    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84f2fd48-b014-4d96-b247-44a77932b73c-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gcbd8\" (UID: \"84f2fd48-b014-4d96-b247-44a77932b73c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gcbd8"
Sep 11 11:59:14.394596 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:14.394522    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gfn8\" (UniqueName: \"kubernetes.io/projected/84f2fd48-b014-4d96-b247-44a77932b73c-kube-api-access-6gfn8\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gcbd8\" (UID: \"84f2fd48-b014-4d96-b247-44a77932b73c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gcbd8"
Sep 11 11:59:14.394719 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:14.394653    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84f2fd48-b014-4d96-b247-44a77932b73c-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gcbd8\" (UID: \"84f2fd48-b014-4d96-b247-44a77932b73c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gcbd8"
Sep 11 11:59:14.394719 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:14.394711    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84f2fd48-b014-4d96-b247-44a77932b73c-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gcbd8\" (UID: \"84f2fd48-b014-4d96-b247-44a77932b73c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gcbd8"
Sep 11 11:59:14.394835 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:14.394809    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84f2fd48-b014-4d96-b247-44a77932b73c-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gcbd8\" (UID: \"84f2fd48-b014-4d96-b247-44a77932b73c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gcbd8"
Sep 11 11:59:14.394893 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:14.394871    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84f2fd48-b014-4d96-b247-44a77932b73c-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gcbd8\" (UID: \"84f2fd48-b014-4d96-b247-44a77932b73c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gcbd8"
Sep 11 11:59:14.394996 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:14.394977    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84f2fd48-b014-4d96-b247-44a77932b73c-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gcbd8\" (UID: \"84f2fd48-b014-4d96-b247-44a77932b73c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gcbd8"
Sep 11 11:59:14.396683 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:14.396662    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84f2fd48-b014-4d96-b247-44a77932b73c-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gcbd8\" (UID: \"84f2fd48-b014-4d96-b247-44a77932b73c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gcbd8"
Sep 11 11:59:14.397110 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:14.397092    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84f2fd48-b014-4d96-b247-44a77932b73c-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gcbd8\" (UID: \"84f2fd48-b014-4d96-b247-44a77932b73c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gcbd8"
Sep 11 11:59:14.406172 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:14.406142    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gfn8\" (UniqueName: \"kubernetes.io/projected/84f2fd48-b014-4d96-b247-44a77932b73c-kube-api-access-6gfn8\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gcbd8\" (UID: \"84f2fd48-b014-4d96-b247-44a77932b73c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gcbd8"
Sep 11 11:59:14.556682 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:14.556597    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gcbd8"
Sep 11 11:59:14.832677 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:14.832488    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-gcbd8"]
Sep 11 11:59:14.835443 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:59:14.835418    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84f2fd48_b014_4d96_b247_44a77932b73c.slice/crio-109e86cc66b1e5314229a78d1365b091135fc88d9b0a0f8671b66db0dd060358 WatchSource:0}: Error finding container 109e86cc66b1e5314229a78d1365b091135fc88d9b0a0f8671b66db0dd060358: Status 404 returned error can't find the container with id 109e86cc66b1e5314229a78d1365b091135fc88d9b0a0f8671b66db0dd060358
Sep 11 11:59:15.002875 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:15.002838    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 11:59:15.515429 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:15.515378    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gcbd8" event={"ID":"84f2fd48-b014-4d96-b247-44a77932b73c","Type":"ContainerStarted","Data":"109e86cc66b1e5314229a78d1365b091135fc88d9b0a0f8671b66db0dd060358"}
Sep 11 11:59:16.521719 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:16.521685    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gcbd8" event={"ID":"84f2fd48-b014-4d96-b247-44a77932b73c","Type":"ContainerStarted","Data":"fab59b7cbf4888b998699df5a303a1d88b32383323ee26d6e437a4579a37f1e7"}
Sep 11 11:59:16.541914 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:16.541860    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gcbd8" podStartSLOduration=1.557069088 podStartE2EDuration="2.541843251s" totalImagesPullingTime="984.774163ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:59:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:59:16.540573566 +0000 UTC m=+660.438015323" watchObservedRunningTime="2026-09-11 11:59:16.541843251 +0000 UTC m=+660.439285029"
Sep 11 11:59:22.556891 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:22.556860    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:59:23.665558 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:23.665526    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 11:59:24.556949 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:24.556908    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gcbd8"
Sep 11 11:59:24.557159 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:24.556983    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gcbd8"
Sep 11 11:59:24.575929 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:24.575901    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gcbd8"
Sep 11 11:59:25.578267 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:25.578238    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gcbd8"
Sep 11 11:59:30.955531 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:30.955494    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:59:32.272914 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:32.272871    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 11:59:43.662633 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:43.662595    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58ppm"]
Sep 11 11:59:43.666246 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:43.666228    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58ppm"
Sep 11 11:59:43.669655 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:43.669636    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Sep 11 11:59:43.678456 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:43.678426    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58ppm"]
Sep 11 11:59:43.691499 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:43.691460    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 11:59:43.771850 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:43.771808    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c868518c-3ff5-46e5-8186-a244b09bd377-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58ppm\" (UID: \"c868518c-3ff5-46e5-8186-a244b09bd377\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58ppm"
Sep 11 11:59:43.772041 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:43.771855    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c868518c-3ff5-46e5-8186-a244b09bd377-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58ppm\" (UID: \"c868518c-3ff5-46e5-8186-a244b09bd377\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58ppm"
Sep 11 11:59:43.772041 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:43.771917    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c868518c-3ff5-46e5-8186-a244b09bd377-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58ppm\" (UID: \"c868518c-3ff5-46e5-8186-a244b09bd377\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58ppm"
Sep 11 11:59:43.772041 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:43.771959    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c868518c-3ff5-46e5-8186-a244b09bd377-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58ppm\" (UID: \"c868518c-3ff5-46e5-8186-a244b09bd377\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58ppm"
Sep 11 11:59:43.772041 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:43.771986    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c868518c-3ff5-46e5-8186-a244b09bd377-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58ppm\" (UID: \"c868518c-3ff5-46e5-8186-a244b09bd377\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58ppm"
Sep 11 11:59:43.772223 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:43.772068    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xn75\" (UniqueName: \"kubernetes.io/projected/c868518c-3ff5-46e5-8186-a244b09bd377-kube-api-access-9xn75\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58ppm\" (UID: \"c868518c-3ff5-46e5-8186-a244b09bd377\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58ppm"
Sep 11 11:59:43.872630 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:43.872590    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xn75\" (UniqueName: \"kubernetes.io/projected/c868518c-3ff5-46e5-8186-a244b09bd377-kube-api-access-9xn75\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58ppm\" (UID: \"c868518c-3ff5-46e5-8186-a244b09bd377\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58ppm"
Sep 11 11:59:43.872630 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:43.872634    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c868518c-3ff5-46e5-8186-a244b09bd377-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58ppm\" (UID: \"c868518c-3ff5-46e5-8186-a244b09bd377\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58ppm"
Sep 11 11:59:43.872888 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:43.872651    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c868518c-3ff5-46e5-8186-a244b09bd377-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58ppm\" (UID: \"c868518c-3ff5-46e5-8186-a244b09bd377\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58ppm"
Sep 11 11:59:43.872888 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:43.872692    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c868518c-3ff5-46e5-8186-a244b09bd377-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58ppm\" (UID: \"c868518c-3ff5-46e5-8186-a244b09bd377\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58ppm"
Sep 11 11:59:43.872888 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:43.872710    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c868518c-3ff5-46e5-8186-a244b09bd377-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58ppm\" (UID: \"c868518c-3ff5-46e5-8186-a244b09bd377\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58ppm"
Sep 11 11:59:43.872888 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:43.872726    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c868518c-3ff5-46e5-8186-a244b09bd377-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58ppm\" (UID: \"c868518c-3ff5-46e5-8186-a244b09bd377\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58ppm"
Sep 11 11:59:43.873107 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:43.873007    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c868518c-3ff5-46e5-8186-a244b09bd377-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58ppm\" (UID: \"c868518c-3ff5-46e5-8186-a244b09bd377\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58ppm"
Sep 11 11:59:43.873107 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:43.873056    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c868518c-3ff5-46e5-8186-a244b09bd377-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58ppm\" (UID: \"c868518c-3ff5-46e5-8186-a244b09bd377\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58ppm"
Sep 11 11:59:43.873209 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:43.873148    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c868518c-3ff5-46e5-8186-a244b09bd377-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58ppm\" (UID: \"c868518c-3ff5-46e5-8186-a244b09bd377\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58ppm"
Sep 11 11:59:43.875097 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:43.875078    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c868518c-3ff5-46e5-8186-a244b09bd377-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58ppm\" (UID: \"c868518c-3ff5-46e5-8186-a244b09bd377\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58ppm"
Sep 11 11:59:43.875265 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:43.875250    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c868518c-3ff5-46e5-8186-a244b09bd377-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58ppm\" (UID: \"c868518c-3ff5-46e5-8186-a244b09bd377\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58ppm"
Sep 11 11:59:43.884014 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:43.883988    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xn75\" (UniqueName: \"kubernetes.io/projected/c868518c-3ff5-46e5-8186-a244b09bd377-kube-api-access-9xn75\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58ppm\" (UID: \"c868518c-3ff5-46e5-8186-a244b09bd377\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58ppm"
Sep 11 11:59:43.976944 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:43.976855    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58ppm"
Sep 11 11:59:44.256532 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:44.256497    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58ppm"]
Sep 11 11:59:44.259126 ip-10-0-142-145 kubenswrapper[2579]: W0911 11:59:44.259096    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc868518c_3ff5_46e5_8186_a244b09bd377.slice/crio-f8689ff4286a737c0424db25f28679ede21569b433bfdc000730e4ce46107f79 WatchSource:0}: Error finding container f8689ff4286a737c0424db25f28679ede21569b433bfdc000730e4ce46107f79: Status 404 returned error can't find the container with id f8689ff4286a737c0424db25f28679ede21569b433bfdc000730e4ce46107f79
Sep 11 11:59:44.672034 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:44.671987    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58ppm" event={"ID":"c868518c-3ff5-46e5-8186-a244b09bd377","Type":"ContainerStarted","Data":"18e257a64e353ea1017de554401ae83a178309b83ef293c606daacd42f8a10b5"}
Sep 11 11:59:44.672528 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:44.672084    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58ppm" event={"ID":"c868518c-3ff5-46e5-8186-a244b09bd377","Type":"ContainerStarted","Data":"f8689ff4286a737c0424db25f28679ede21569b433bfdc000730e4ce46107f79"}
Sep 11 11:59:44.698235 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:44.698178    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58ppm" podStartSLOduration=1.504543843 podStartE2EDuration="1.698162728s" totalImagesPullingTime="193.618885ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:59:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:59:44.697464449 +0000 UTC m=+688.594906196" watchObservedRunningTime="2026-09-11 11:59:44.698162728 +0000 UTC m=+688.595604465"
Sep 11 11:59:45.170536 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:45.170494    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 11:59:53.977055 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:53.977016    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58ppm"
Sep 11 11:59:53.977520 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:53.977090    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58ppm"
Sep 11 11:59:53.990423 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:53.990383    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58ppm"
Sep 11 11:59:54.733871 ip-10-0-142-145 kubenswrapper[2579]: I0911 11:59:54.733838    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58ppm"
Sep 11 12:00:00.144639 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:00:00.144599    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29818800-xtcbc"]
Sep 11 12:00:00.149405 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:00:00.149376    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818800-xtcbc"
Sep 11 12:00:00.152153 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:00:00.152133    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9xkpf\""
Sep 11 12:00:00.156547 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:00:00.156509    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29818800-xtcbc"]
Sep 11 12:00:00.161729 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:00:00.161711    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818800-xtcbc"
Sep 11 12:00:00.458550 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:00:00.458471    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29818800-xtcbc"]
Sep 11 12:00:00.461702 ip-10-0-142-145 kubenswrapper[2579]: W0911 12:00:00.461672    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94cb2699_87c3_4efa_bedf_75341e1bd10f.slice/crio-bf9ec7ae5ad1f22f67bda02f3b00491259f0707b1d56b3e67d7939096b29a9f0 WatchSource:0}: Error finding container bf9ec7ae5ad1f22f67bda02f3b00491259f0707b1d56b3e67d7939096b29a9f0: Status 404 returned error can't find the container with id bf9ec7ae5ad1f22f67bda02f3b00491259f0707b1d56b3e67d7939096b29a9f0
Sep 11 12:00:00.463822 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:00:00.463797    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 12:00:00.755444 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:00:00.755331    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818800-xtcbc" event={"ID":"94cb2699-87c3-4efa-bedf-75341e1bd10f","Type":"ContainerStarted","Data":"bf9ec7ae5ad1f22f67bda02f3b00491259f0707b1d56b3e67d7939096b29a9f0"}
Sep 11 12:00:05.474309 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:00:05.474273    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:00:05.785202 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:00:05.785107    2579 generic.go:356] "Generic (PLEG): container finished" podID="94cb2699-87c3-4efa-bedf-75341e1bd10f" containerID="1fc416c5921174abd75e8314b88a4092183346bb08a2cb07f335094501329a33" exitCode=0
Sep 11 12:00:05.785360 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:00:05.785194    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818800-xtcbc" event={"ID":"94cb2699-87c3-4efa-bedf-75341e1bd10f","Type":"ContainerDied","Data":"1fc416c5921174abd75e8314b88a4092183346bb08a2cb07f335094501329a33"}
Sep 11 12:00:06.921590 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:00:06.921563    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818800-xtcbc"
Sep 11 12:00:07.796362 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:00:07.796330    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818800-xtcbc"
Sep 11 12:00:07.796550 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:00:07.796369    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818800-xtcbc" event={"ID":"94cb2699-87c3-4efa-bedf-75341e1bd10f","Type":"ContainerDied","Data":"bf9ec7ae5ad1f22f67bda02f3b00491259f0707b1d56b3e67d7939096b29a9f0"}
Sep 11 12:00:07.796550 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:00:07.796422    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bf9ec7ae5ad1f22f67bda02f3b00491259f0707b1d56b3e67d7939096b29a9f0"
Sep 11 12:00:17.194299 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:00:17.194215    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:00:19.766134 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:00:19.766104    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:00:32.639515 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:00:32.639462    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:00:34.775752 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:00:34.775712    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:00:48.185104 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:00:48.185074    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:00:50.270677 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:00:50.270640    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:01:09.953691 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:01:09.953650    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:01:51.684041 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:01:51.683928    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:01:51.776244 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:01:51.776204    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:01:52.854595 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:01:52.854560    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:01:53.048835 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:01:53.048789    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:08.268720 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:08.268685    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:08.823760 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:08.823720    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:09.403635 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:09.403595    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:09.503855 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:09.503817    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:09.732433 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:09.732324    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:09.851633 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:09.851595    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:10.122585 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:10.122548    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:12.068582 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:12.068545    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:02:12.210311 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:12.210274    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:12.318095 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:12.318064    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:13.762584 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:13.762529    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:13.859852 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:13.859816    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:13.960771 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:13.960734    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:14.286968 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:14.286937    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-cbc380-d9ccbfcb7-zrzjq"]
Sep 11 12:02:14.287337 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:14.287323    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94cb2699-87c3-4efa-bedf-75341e1bd10f" containerName="cleanup"
Sep 11 12:02:14.287415 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:14.287339    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="94cb2699-87c3-4efa-bedf-75341e1bd10f" containerName="cleanup"
Sep 11 12:02:14.287458 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:14.287430    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="94cb2699-87c3-4efa-bedf-75341e1bd10f" containerName="cleanup"
Sep 11 12:02:14.290660 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:14.290640    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-cbc380-d9ccbfcb7-zrzjq"
Sep 11 12:02:14.293687 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:14.293654    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-cbc380\""
Sep 11 12:02:14.293818 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:14.293756    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-cbc380-dockercfg-9rdhw\""
Sep 11 12:02:14.304613 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:14.304576    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-cbc380-d9ccbfcb7-zrzjq"]
Sep 11 12:02:14.381785 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:14.381755    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f7a23ed-b195-4dcb-a35a-466ebde97aae-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-cbc380-d9ccbfcb7-zrzjq\" (UID: \"2f7a23ed-b195-4dcb-a35a-466ebde97aae\") " pod="openshift-ingress/payload-processing-e2e-inf-a-cbc380-d9ccbfcb7-zrzjq"
Sep 11 12:02:14.381975 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:14.381798    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wd4sr\" (UniqueName: \"kubernetes.io/projected/2f7a23ed-b195-4dcb-a35a-466ebde97aae-kube-api-access-wd4sr\") pod \"payload-processing-e2e-inf-a-cbc380-d9ccbfcb7-zrzjq\" (UID: \"2f7a23ed-b195-4dcb-a35a-466ebde97aae\") " pod="openshift-ingress/payload-processing-e2e-inf-a-cbc380-d9ccbfcb7-zrzjq"
Sep 11 12:02:14.482822 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:14.482778    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wd4sr\" (UniqueName: \"kubernetes.io/projected/2f7a23ed-b195-4dcb-a35a-466ebde97aae-kube-api-access-wd4sr\") pod \"payload-processing-e2e-inf-a-cbc380-d9ccbfcb7-zrzjq\" (UID: \"2f7a23ed-b195-4dcb-a35a-466ebde97aae\") " pod="openshift-ingress/payload-processing-e2e-inf-a-cbc380-d9ccbfcb7-zrzjq"
Sep 11 12:02:14.483017 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:14.482940    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f7a23ed-b195-4dcb-a35a-466ebde97aae-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-cbc380-d9ccbfcb7-zrzjq\" (UID: \"2f7a23ed-b195-4dcb-a35a-466ebde97aae\") " pod="openshift-ingress/payload-processing-e2e-inf-a-cbc380-d9ccbfcb7-zrzjq"
Sep 11 12:02:14.483721 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:14.483700    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f7a23ed-b195-4dcb-a35a-466ebde97aae-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-cbc380-d9ccbfcb7-zrzjq\" (UID: \"2f7a23ed-b195-4dcb-a35a-466ebde97aae\") " pod="openshift-ingress/payload-processing-e2e-inf-a-cbc380-d9ccbfcb7-zrzjq"
Sep 11 12:02:14.496612 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:14.496573    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wd4sr\" (UniqueName: \"kubernetes.io/projected/2f7a23ed-b195-4dcb-a35a-466ebde97aae-kube-api-access-wd4sr\") pod \"payload-processing-e2e-inf-a-cbc380-d9ccbfcb7-zrzjq\" (UID: \"2f7a23ed-b195-4dcb-a35a-466ebde97aae\") " pod="openshift-ingress/payload-processing-e2e-inf-a-cbc380-d9ccbfcb7-zrzjq"
Sep 11 12:02:14.536603 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:14.536560    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:14.601893 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:14.601857    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-cbc380-d9ccbfcb7-zrzjq"
Sep 11 12:02:14.665714 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:14.665622    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:14.877019 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:14.876988    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-cbc380-d9ccbfcb7-zrzjq"]
Sep 11 12:02:14.879069 ip-10-0-142-145 kubenswrapper[2579]: W0911 12:02:14.879040    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f7a23ed_b195_4dcb_a35a_466ebde97aae.slice/crio-83e28c0a2d0ce0494359b5687b77ae5ed9a47fa27e5837dbc052e596cec2ed63 WatchSource:0}: Error finding container 83e28c0a2d0ce0494359b5687b77ae5ed9a47fa27e5837dbc052e596cec2ed63: Status 404 returned error can't find the container with id 83e28c0a2d0ce0494359b5687b77ae5ed9a47fa27e5837dbc052e596cec2ed63
Sep 11 12:02:15.436778 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:15.436741    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-cbc380-d9ccbfcb7-zrzjq" event={"ID":"2f7a23ed-b195-4dcb-a35a-466ebde97aae","Type":"ContainerStarted","Data":"83e28c0a2d0ce0494359b5687b77ae5ed9a47fa27e5837dbc052e596cec2ed63"}
Sep 11 12:02:17.446148 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:17.446112    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:17.585532 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:17.585490    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:17.707620 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:17.707525    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:18.113702 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:18.113652    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:18.384987 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:18.384887    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:18.455785 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:18.455751    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-cbc380-d9ccbfcb7-zrzjq" event={"ID":"2f7a23ed-b195-4dcb-a35a-466ebde97aae","Type":"ContainerStarted","Data":"0741a9251bc6c03ddaebb186f04958c24d2fa6ec509d0d3980e6163b212e98cd"}
Sep 11 12:02:18.456176 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:18.455892    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-cbc380-d9ccbfcb7-zrzjq"
Sep 11 12:02:18.492801 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:18.492745    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-cbc380-d9ccbfcb7-zrzjq" podStartSLOduration=1.6395789079999998 podStartE2EDuration="4.492727777s" totalImagesPullingTime="2.853148869s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:02:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:02:18.491999777 +0000 UTC m=+842.389441521" watchObservedRunningTime="2026-09-11 12:02:18.492727777 +0000 UTC m=+842.390169563"
Sep 11 12:02:18.887656 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:18.887619    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:21.376909 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:21.376870    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:21.593930 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:21.593892    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:22.470269 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:22.470233    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-cdec04-585c76b54c-544tc"]
Sep 11 12:02:22.474007 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:22.473982    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-cdec04-585c76b54c-544tc"
Sep 11 12:02:22.482491 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:22.482460    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 11 12:02:22.482658 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:22.482550    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-cdec04-serving-cert\""
Sep 11 12:02:22.483570 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:22.483548    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 11 12:02:22.483679 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:22.483598    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-rrxjh\""
Sep 11 12:02:22.483872 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:22.483856    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 12:02:22.505102 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:22.505067    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-cdec04-585c76b54c-544tc"]
Sep 11 12:02:22.538764 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:22.538730    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:22.560615 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:22.560585    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/429c86e4-a876-4029-9eaa-264a892cef11-openshift-service-ca\") pod \"maas-api-e2e-autores-cdec04-585c76b54c-544tc\" (UID: \"429c86e4-a876-4029-9eaa-264a892cef11\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-cdec04-585c76b54c-544tc"
Sep 11 12:02:22.560615 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:22.560621    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/429c86e4-a876-4029-9eaa-264a892cef11-maas-api-tls\") pod \"maas-api-e2e-autores-cdec04-585c76b54c-544tc\" (UID: \"429c86e4-a876-4029-9eaa-264a892cef11\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-cdec04-585c76b54c-544tc"
Sep 11 12:02:22.560820 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:22.560642    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/429c86e4-a876-4029-9eaa-264a892cef11-metrics-certs\") pod \"maas-api-e2e-autores-cdec04-585c76b54c-544tc\" (UID: \"429c86e4-a876-4029-9eaa-264a892cef11\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-cdec04-585c76b54c-544tc"
Sep 11 12:02:22.560820 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:22.560732    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxswk\" (UniqueName: \"kubernetes.io/projected/429c86e4-a876-4029-9eaa-264a892cef11-kube-api-access-zxswk\") pod \"maas-api-e2e-autores-cdec04-585c76b54c-544tc\" (UID: \"429c86e4-a876-4029-9eaa-264a892cef11\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-cdec04-585c76b54c-544tc"
Sep 11 12:02:22.660845 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:22.660811    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:22.661217 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:22.661196    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/429c86e4-a876-4029-9eaa-264a892cef11-openshift-service-ca\") pod \"maas-api-e2e-autores-cdec04-585c76b54c-544tc\" (UID: \"429c86e4-a876-4029-9eaa-264a892cef11\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-cdec04-585c76b54c-544tc"
Sep 11 12:02:22.661321 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:22.661230    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/429c86e4-a876-4029-9eaa-264a892cef11-maas-api-tls\") pod \"maas-api-e2e-autores-cdec04-585c76b54c-544tc\" (UID: \"429c86e4-a876-4029-9eaa-264a892cef11\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-cdec04-585c76b54c-544tc"
Sep 11 12:02:22.661321 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:22.661255    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/429c86e4-a876-4029-9eaa-264a892cef11-metrics-certs\") pod \"maas-api-e2e-autores-cdec04-585c76b54c-544tc\" (UID: \"429c86e4-a876-4029-9eaa-264a892cef11\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-cdec04-585c76b54c-544tc"
Sep 11 12:02:22.661321 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:22.661290    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxswk\" (UniqueName: \"kubernetes.io/projected/429c86e4-a876-4029-9eaa-264a892cef11-kube-api-access-zxswk\") pod \"maas-api-e2e-autores-cdec04-585c76b54c-544tc\" (UID: \"429c86e4-a876-4029-9eaa-264a892cef11\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-cdec04-585c76b54c-544tc"
Sep 11 12:02:22.661484 ip-10-0-142-145 kubenswrapper[2579]: E0911 12:02:22.661356    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-autores-cdec04-serving-cert: secret "maas-api-e2e-autores-cdec04-serving-cert" not found
Sep 11 12:02:22.661484 ip-10-0-142-145 kubenswrapper[2579]: E0911 12:02:22.661455    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/429c86e4-a876-4029-9eaa-264a892cef11-maas-api-tls podName:429c86e4-a876-4029-9eaa-264a892cef11 nodeName:}" failed. No retries permitted until 2026-09-11 12:02:23.161436213 +0000 UTC m=+847.058877953 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/429c86e4-a876-4029-9eaa-264a892cef11-maas-api-tls") pod "maas-api-e2e-autores-cdec04-585c76b54c-544tc" (UID: "429c86e4-a876-4029-9eaa-264a892cef11") : secret "maas-api-e2e-autores-cdec04-serving-cert" not found
Sep 11 12:02:22.661949 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:22.661918    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/429c86e4-a876-4029-9eaa-264a892cef11-openshift-service-ca\") pod \"maas-api-e2e-autores-cdec04-585c76b54c-544tc\" (UID: \"429c86e4-a876-4029-9eaa-264a892cef11\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-cdec04-585c76b54c-544tc"
Sep 11 12:02:22.663767 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:22.663746    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/429c86e4-a876-4029-9eaa-264a892cef11-metrics-certs\") pod \"maas-api-e2e-autores-cdec04-585c76b54c-544tc\" (UID: \"429c86e4-a876-4029-9eaa-264a892cef11\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-cdec04-585c76b54c-544tc"
Sep 11 12:02:22.687606 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:22.687570    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxswk\" (UniqueName: \"kubernetes.io/projected/429c86e4-a876-4029-9eaa-264a892cef11-kube-api-access-zxswk\") pod \"maas-api-e2e-autores-cdec04-585c76b54c-544tc\" (UID: \"429c86e4-a876-4029-9eaa-264a892cef11\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-cdec04-585c76b54c-544tc"
Sep 11 12:02:22.849630 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:22.849591    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:23.166117 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:23.166026    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/429c86e4-a876-4029-9eaa-264a892cef11-maas-api-tls\") pod \"maas-api-e2e-autores-cdec04-585c76b54c-544tc\" (UID: \"429c86e4-a876-4029-9eaa-264a892cef11\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-cdec04-585c76b54c-544tc"
Sep 11 12:02:23.168734 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:23.168677    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/429c86e4-a876-4029-9eaa-264a892cef11-maas-api-tls\") pod \"maas-api-e2e-autores-cdec04-585c76b54c-544tc\" (UID: \"429c86e4-a876-4029-9eaa-264a892cef11\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-cdec04-585c76b54c-544tc"
Sep 11 12:02:23.249180 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:23.249144    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:23.365324 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:23.365286    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:23.387454 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:23.387418    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-cdec04-585c76b54c-544tc"
Sep 11 12:02:23.669794 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:23.669709    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-cdec04-585c76b54c-544tc"]
Sep 11 12:02:23.672584 ip-10-0-142-145 kubenswrapper[2579]: W0911 12:02:23.672553    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod429c86e4_a876_4029_9eaa_264a892cef11.slice/crio-2f9f75c0cc084e525fd1536fe8252a17ab3389e7533353865638afa12cef72ad WatchSource:0}: Error finding container 2f9f75c0cc084e525fd1536fe8252a17ab3389e7533353865638afa12cef72ad: Status 404 returned error can't find the container with id 2f9f75c0cc084e525fd1536fe8252a17ab3389e7533353865638afa12cef72ad
Sep 11 12:02:23.924052 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:23.924015    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:24.110699 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:24.110662    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:24.492470 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:24.492429    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-cdec04-585c76b54c-544tc" event={"ID":"429c86e4-a876-4029-9eaa-264a892cef11","Type":"ContainerStarted","Data":"2f9f75c0cc084e525fd1536fe8252a17ab3389e7533353865638afa12cef72ad"}
Sep 11 12:02:25.374829 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:25.374790    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:25.481856 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:25.481815    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:25.798492 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:25.798404    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:26.073875 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:26.073774    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:26.193888 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:26.193849    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:26.505387 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:26.505348    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-cdec04-585c76b54c-544tc" event={"ID":"429c86e4-a876-4029-9eaa-264a892cef11","Type":"ContainerStarted","Data":"ff365fba54a79fcb6b33b50a13f757af2b20522373f5a164d3d9c91c911dd5b4"}
Sep 11 12:02:26.505782 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:26.505436    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-cdec04-585c76b54c-544tc"
Sep 11 12:02:26.534964 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:26.534912    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-cdec04-585c76b54c-544tc" podStartSLOduration=2.105362607 podStartE2EDuration="4.534893808s" totalImagesPullingTime="2.429531201s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:02:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:02:26.533051387 +0000 UTC m=+850.430493132" watchObservedRunningTime="2026-09-11 12:02:26.534893808 +0000 UTC m=+850.432335747"
Sep 11 12:02:26.683167 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:26.683115    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:27.503287 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:27.503250    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:27.551478 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:27.551438    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:02:27.674678 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:27.674634    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:27.818649 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:27.818552    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:28.150273 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:28.150232    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:29.463577 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:29.463543    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-cbc380-d9ccbfcb7-zrzjq"
Sep 11 12:02:30.479204 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:30.479169    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:31.832273 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:31.832233    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:31.931078 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:31.931038    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:32.022087 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:32.022056    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:32.516404 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:32.516365    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-cdec04-585c76b54c-544tc"
Sep 11 12:02:32.849837 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:32.849812    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:34.658660 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:34.658618    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:36.993904 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:36.993872    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:37.647928 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:37.647894    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv"]
Sep 11 12:02:37.651805 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:37.651781    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv"
Sep 11 12:02:37.654590 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:37.654569    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-12d2a9-openshift-default-dockercfg-2rwgl\""
Sep 11 12:02:37.666096 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:37.666067    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv"]
Sep 11 12:02:37.682405 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:37.682364    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:37.709966 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:37.709919    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36e69a19-0f39-41aa-b80e-e60b2b9cf3c7-istio-token\") pod \"e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv\" (UID: \"36e69a19-0f39-41aa-b80e-e60b2b9cf3c7\") " pod="openshift-ingress/e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv"
Sep 11 12:02:37.710152 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:37.709983    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36e69a19-0f39-41aa-b80e-e60b2b9cf3c7-workload-certs\") pod \"e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv\" (UID: \"36e69a19-0f39-41aa-b80e-e60b2b9cf3c7\") " pod="openshift-ingress/e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv"
Sep 11 12:02:37.710152 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:37.710017    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36e69a19-0f39-41aa-b80e-e60b2b9cf3c7-istio-envoy\") pod \"e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv\" (UID: \"36e69a19-0f39-41aa-b80e-e60b2b9cf3c7\") " pod="openshift-ingress/e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv"
Sep 11 12:02:37.710152 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:37.710042    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36e69a19-0f39-41aa-b80e-e60b2b9cf3c7-istio-data\") pod \"e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv\" (UID: \"36e69a19-0f39-41aa-b80e-e60b2b9cf3c7\") " pod="openshift-ingress/e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv"
Sep 11 12:02:37.710152 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:37.710106    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36e69a19-0f39-41aa-b80e-e60b2b9cf3c7-istiod-ca-cert\") pod \"e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv\" (UID: \"36e69a19-0f39-41aa-b80e-e60b2b9cf3c7\") " pod="openshift-ingress/e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv"
Sep 11 12:02:37.710329 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:37.710180    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36e69a19-0f39-41aa-b80e-e60b2b9cf3c7-workload-socket\") pod \"e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv\" (UID: \"36e69a19-0f39-41aa-b80e-e60b2b9cf3c7\") " pod="openshift-ingress/e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv"
Sep 11 12:02:37.710329 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:37.710210    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8vdj\" (UniqueName: \"kubernetes.io/projected/36e69a19-0f39-41aa-b80e-e60b2b9cf3c7-kube-api-access-q8vdj\") pod \"e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv\" (UID: \"36e69a19-0f39-41aa-b80e-e60b2b9cf3c7\") " pod="openshift-ingress/e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv"
Sep 11 12:02:37.710329 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:37.710249    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36e69a19-0f39-41aa-b80e-e60b2b9cf3c7-istio-podinfo\") pod \"e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv\" (UID: \"36e69a19-0f39-41aa-b80e-e60b2b9cf3c7\") " pod="openshift-ingress/e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv"
Sep 11 12:02:37.710482 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:37.710365    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36e69a19-0f39-41aa-b80e-e60b2b9cf3c7-credential-socket\") pod \"e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv\" (UID: \"36e69a19-0f39-41aa-b80e-e60b2b9cf3c7\") " pod="openshift-ingress/e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv"
Sep 11 12:02:37.784738 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:37.784709    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:37.811555 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:37.811525    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36e69a19-0f39-41aa-b80e-e60b2b9cf3c7-credential-socket\") pod \"e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv\" (UID: \"36e69a19-0f39-41aa-b80e-e60b2b9cf3c7\") " pod="openshift-ingress/e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv"
Sep 11 12:02:37.811724 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:37.811580    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36e69a19-0f39-41aa-b80e-e60b2b9cf3c7-istio-token\") pod \"e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv\" (UID: \"36e69a19-0f39-41aa-b80e-e60b2b9cf3c7\") " pod="openshift-ingress/e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv"
Sep 11 12:02:37.811724 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:37.811620    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36e69a19-0f39-41aa-b80e-e60b2b9cf3c7-workload-certs\") pod \"e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv\" (UID: \"36e69a19-0f39-41aa-b80e-e60b2b9cf3c7\") " pod="openshift-ingress/e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv"
Sep 11 12:02:37.811724 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:37.811646    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36e69a19-0f39-41aa-b80e-e60b2b9cf3c7-istio-envoy\") pod \"e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv\" (UID: \"36e69a19-0f39-41aa-b80e-e60b2b9cf3c7\") " pod="openshift-ingress/e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv"
Sep 11 12:02:37.811724 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:37.811670    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36e69a19-0f39-41aa-b80e-e60b2b9cf3c7-istio-data\") pod \"e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv\" (UID: \"36e69a19-0f39-41aa-b80e-e60b2b9cf3c7\") " pod="openshift-ingress/e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv"
Sep 11 12:02:37.811724 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:37.811712    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36e69a19-0f39-41aa-b80e-e60b2b9cf3c7-istiod-ca-cert\") pod \"e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv\" (UID: \"36e69a19-0f39-41aa-b80e-e60b2b9cf3c7\") " pod="openshift-ingress/e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv"
Sep 11 12:02:37.811984 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:37.811757    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36e69a19-0f39-41aa-b80e-e60b2b9cf3c7-workload-socket\") pod \"e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv\" (UID: \"36e69a19-0f39-41aa-b80e-e60b2b9cf3c7\") " pod="openshift-ingress/e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv"
Sep 11 12:02:37.811984 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:37.811782    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8vdj\" (UniqueName: \"kubernetes.io/projected/36e69a19-0f39-41aa-b80e-e60b2b9cf3c7-kube-api-access-q8vdj\") pod \"e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv\" (UID: \"36e69a19-0f39-41aa-b80e-e60b2b9cf3c7\") " pod="openshift-ingress/e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv"
Sep 11 12:02:37.811984 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:37.811819    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36e69a19-0f39-41aa-b80e-e60b2b9cf3c7-istio-podinfo\") pod \"e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv\" (UID: \"36e69a19-0f39-41aa-b80e-e60b2b9cf3c7\") " pod="openshift-ingress/e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv"
Sep 11 12:02:37.812246 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:37.812218    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36e69a19-0f39-41aa-b80e-e60b2b9cf3c7-workload-certs\") pod \"e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv\" (UID: \"36e69a19-0f39-41aa-b80e-e60b2b9cf3c7\") " pod="openshift-ingress/e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv"
Sep 11 12:02:37.812445 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:37.812262    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36e69a19-0f39-41aa-b80e-e60b2b9cf3c7-workload-socket\") pod \"e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv\" (UID: \"36e69a19-0f39-41aa-b80e-e60b2b9cf3c7\") " pod="openshift-ingress/e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv"
Sep 11 12:02:37.812566 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:37.812504    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36e69a19-0f39-41aa-b80e-e60b2b9cf3c7-istio-data\") pod \"e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv\" (UID: \"36e69a19-0f39-41aa-b80e-e60b2b9cf3c7\") " pod="openshift-ingress/e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv"
Sep 11 12:02:37.812566 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:37.812537    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36e69a19-0f39-41aa-b80e-e60b2b9cf3c7-istiod-ca-cert\") pod \"e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv\" (UID: \"36e69a19-0f39-41aa-b80e-e60b2b9cf3c7\") " pod="openshift-ingress/e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv"
Sep 11 12:02:37.812802 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:37.812778    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36e69a19-0f39-41aa-b80e-e60b2b9cf3c7-credential-socket\") pod \"e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv\" (UID: \"36e69a19-0f39-41aa-b80e-e60b2b9cf3c7\") " pod="openshift-ingress/e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv"
Sep 11 12:02:37.814671 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:37.814646    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36e69a19-0f39-41aa-b80e-e60b2b9cf3c7-istio-envoy\") pod \"e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv\" (UID: \"36e69a19-0f39-41aa-b80e-e60b2b9cf3c7\") " pod="openshift-ingress/e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv"
Sep 11 12:02:37.814671 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:37.814659    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36e69a19-0f39-41aa-b80e-e60b2b9cf3c7-istio-podinfo\") pod \"e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv\" (UID: \"36e69a19-0f39-41aa-b80e-e60b2b9cf3c7\") " pod="openshift-ingress/e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv"
Sep 11 12:02:37.822236 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:37.822203    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8vdj\" (UniqueName: \"kubernetes.io/projected/36e69a19-0f39-41aa-b80e-e60b2b9cf3c7-kube-api-access-q8vdj\") pod \"e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv\" (UID: \"36e69a19-0f39-41aa-b80e-e60b2b9cf3c7\") " pod="openshift-ingress/e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv"
Sep 11 12:02:37.823367 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:37.823348    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36e69a19-0f39-41aa-b80e-e60b2b9cf3c7-istio-token\") pod \"e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv\" (UID: \"36e69a19-0f39-41aa-b80e-e60b2b9cf3c7\") " pod="openshift-ingress/e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv"
Sep 11 12:02:37.962632 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:37.962537    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv"
Sep 11 12:02:38.258631 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:38.258595    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv"]
Sep 11 12:02:38.260314 ip-10-0-142-145 kubenswrapper[2579]: W0911 12:02:38.260286    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36e69a19_0f39_41aa_b80e_e60b2b9cf3c7.slice/crio-2398fed4df503a7aaa893138fe31f0a2de7e7e9287692e5275787e2481a4e6c7 WatchSource:0}: Error finding container 2398fed4df503a7aaa893138fe31f0a2de7e7e9287692e5275787e2481a4e6c7: Status 404 returned error can't find the container with id 2398fed4df503a7aaa893138fe31f0a2de7e7e9287692e5275787e2481a4e6c7
Sep 11 12:02:38.568021 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:38.567927    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv" event={"ID":"36e69a19-0f39-41aa-b80e-e60b2b9cf3c7","Type":"ContainerStarted","Data":"2398fed4df503a7aaa893138fe31f0a2de7e7e9287692e5275787e2481a4e6c7"}
Sep 11 12:02:38.684596 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:38.684564    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:40.378684 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:40.378645    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:40.496264 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:40.496214    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:40.634034 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:40.633942    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:44.269686 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:44.269647    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 12:02:44.269968 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:44.269724    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 12:02:44.269968 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:44.269758    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 12:02:44.601896 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:44.601858    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv" event={"ID":"36e69a19-0f39-41aa-b80e-e60b2b9cf3c7","Type":"ContainerStarted","Data":"e5491aa364bf970cb3d5d9121e4da22b60df276ed7520131ef95cbeceb42b291"}
Sep 11 12:02:44.632584 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:44.632528    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv" podStartSLOduration=1.625629337 podStartE2EDuration="7.632509675s" totalImagesPullingTime="6.006880338s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:02:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:02:44.63090076 +0000 UTC m=+868.528342509" watchObservedRunningTime="2026-09-11 12:02:44.632509675 +0000 UTC m=+868.529951421"
Sep 11 12:02:44.963504 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:44.963407    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv"
Sep 11 12:02:45.967863 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:45.967833    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv"
Sep 11 12:02:46.581972 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:46.581937    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:46.613424 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:46.613376    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv"
Sep 11 12:02:46.614335 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:46.614316    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv"
Sep 11 12:02:46.695667 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:46.695620    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:46.848308 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:46.848248    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:47.592127 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:47.592081    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:02:48.036857 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:48.036823    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-12d2a9-57b894675b-phbq4"]
Sep 11 12:02:48.043326 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:48.043298    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-12d2a9-57b894675b-phbq4"
Sep 11 12:02:48.046431 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:48.046385    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-12d2a9-serving-cert\""
Sep 11 12:02:48.056670 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:48.056638    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-12d2a9-57b894675b-phbq4"]
Sep 11 12:02:48.073725 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:48.073688    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:48.113225 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:48.113185    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2418f3a0-c6e5-4a97-b33d-f5f62fd70741-openshift-service-ca\") pod \"maas-api-e2e-inf-b-12d2a9-57b894675b-phbq4\" (UID: \"2418f3a0-c6e5-4a97-b33d-f5f62fd70741\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-12d2a9-57b894675b-phbq4"
Sep 11 12:02:48.113459 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:48.113279    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2418f3a0-c6e5-4a97-b33d-f5f62fd70741-maas-api-tls\") pod \"maas-api-e2e-inf-b-12d2a9-57b894675b-phbq4\" (UID: \"2418f3a0-c6e5-4a97-b33d-f5f62fd70741\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-12d2a9-57b894675b-phbq4"
Sep 11 12:02:48.113459 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:48.113347    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bs5z5\" (UniqueName: \"kubernetes.io/projected/2418f3a0-c6e5-4a97-b33d-f5f62fd70741-kube-api-access-bs5z5\") pod \"maas-api-e2e-inf-b-12d2a9-57b894675b-phbq4\" (UID: \"2418f3a0-c6e5-4a97-b33d-f5f62fd70741\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-12d2a9-57b894675b-phbq4"
Sep 11 12:02:48.113459 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:48.113387    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2418f3a0-c6e5-4a97-b33d-f5f62fd70741-metrics-certs\") pod \"maas-api-e2e-inf-b-12d2a9-57b894675b-phbq4\" (UID: \"2418f3a0-c6e5-4a97-b33d-f5f62fd70741\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-12d2a9-57b894675b-phbq4"
Sep 11 12:02:48.179827 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:48.179788    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:48.214686 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:48.214654    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2418f3a0-c6e5-4a97-b33d-f5f62fd70741-metrics-certs\") pod \"maas-api-e2e-inf-b-12d2a9-57b894675b-phbq4\" (UID: \"2418f3a0-c6e5-4a97-b33d-f5f62fd70741\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-12d2a9-57b894675b-phbq4"
Sep 11 12:02:48.214858 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:48.214766    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2418f3a0-c6e5-4a97-b33d-f5f62fd70741-openshift-service-ca\") pod \"maas-api-e2e-inf-b-12d2a9-57b894675b-phbq4\" (UID: \"2418f3a0-c6e5-4a97-b33d-f5f62fd70741\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-12d2a9-57b894675b-phbq4"
Sep 11 12:02:48.214858 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:48.214792    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2418f3a0-c6e5-4a97-b33d-f5f62fd70741-maas-api-tls\") pod \"maas-api-e2e-inf-b-12d2a9-57b894675b-phbq4\" (UID: \"2418f3a0-c6e5-4a97-b33d-f5f62fd70741\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-12d2a9-57b894675b-phbq4"
Sep 11 12:02:48.214858 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:48.214825    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bs5z5\" (UniqueName: \"kubernetes.io/projected/2418f3a0-c6e5-4a97-b33d-f5f62fd70741-kube-api-access-bs5z5\") pod \"maas-api-e2e-inf-b-12d2a9-57b894675b-phbq4\" (UID: \"2418f3a0-c6e5-4a97-b33d-f5f62fd70741\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-12d2a9-57b894675b-phbq4"
Sep 11 12:02:48.214979 ip-10-0-142-145 kubenswrapper[2579]: E0911 12:02:48.214920    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-12d2a9-serving-cert: secret "maas-api-e2e-inf-b-12d2a9-serving-cert" not found
Sep 11 12:02:48.215018 ip-10-0-142-145 kubenswrapper[2579]: E0911 12:02:48.214984    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2418f3a0-c6e5-4a97-b33d-f5f62fd70741-maas-api-tls podName:2418f3a0-c6e5-4a97-b33d-f5f62fd70741 nodeName:}" failed. No retries permitted until 2026-09-11 12:02:48.714968388 +0000 UTC m=+872.612410122 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2418f3a0-c6e5-4a97-b33d-f5f62fd70741-maas-api-tls") pod "maas-api-e2e-inf-b-12d2a9-57b894675b-phbq4" (UID: "2418f3a0-c6e5-4a97-b33d-f5f62fd70741") : secret "maas-api-e2e-inf-b-12d2a9-serving-cert" not found
Sep 11 12:02:48.215498 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:48.215480    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2418f3a0-c6e5-4a97-b33d-f5f62fd70741-openshift-service-ca\") pod \"maas-api-e2e-inf-b-12d2a9-57b894675b-phbq4\" (UID: \"2418f3a0-c6e5-4a97-b33d-f5f62fd70741\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-12d2a9-57b894675b-phbq4"
Sep 11 12:02:48.217262 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:48.217246    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2418f3a0-c6e5-4a97-b33d-f5f62fd70741-metrics-certs\") pod \"maas-api-e2e-inf-b-12d2a9-57b894675b-phbq4\" (UID: \"2418f3a0-c6e5-4a97-b33d-f5f62fd70741\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-12d2a9-57b894675b-phbq4"
Sep 11 12:02:48.242498 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:48.242470    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bs5z5\" (UniqueName: \"kubernetes.io/projected/2418f3a0-c6e5-4a97-b33d-f5f62fd70741-kube-api-access-bs5z5\") pod \"maas-api-e2e-inf-b-12d2a9-57b894675b-phbq4\" (UID: \"2418f3a0-c6e5-4a97-b33d-f5f62fd70741\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-12d2a9-57b894675b-phbq4"
Sep 11 12:02:48.529703 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:48.529671    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:48.719224 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:48.719183    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2418f3a0-c6e5-4a97-b33d-f5f62fd70741-maas-api-tls\") pod \"maas-api-e2e-inf-b-12d2a9-57b894675b-phbq4\" (UID: \"2418f3a0-c6e5-4a97-b33d-f5f62fd70741\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-12d2a9-57b894675b-phbq4"
Sep 11 12:02:48.721641 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:48.721619    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2418f3a0-c6e5-4a97-b33d-f5f62fd70741-maas-api-tls\") pod \"maas-api-e2e-inf-b-12d2a9-57b894675b-phbq4\" (UID: \"2418f3a0-c6e5-4a97-b33d-f5f62fd70741\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-12d2a9-57b894675b-phbq4"
Sep 11 12:02:48.896230 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:48.896196    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:48.954741 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:48.954708    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-12d2a9-57b894675b-phbq4"
Sep 11 12:02:49.038385 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:49.038351    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:49.158029 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:49.157925    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:49.269077 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:49.269056    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-12d2a9-57b894675b-phbq4"]
Sep 11 12:02:49.271505 ip-10-0-142-145 kubenswrapper[2579]: W0911 12:02:49.271475    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2418f3a0_c6e5_4a97_b33d_f5f62fd70741.slice/crio-5e10096c44c423cf3557a51f49135a9bf8b72bea4efed3bdb2b1384502882277 WatchSource:0}: Error finding container 5e10096c44c423cf3557a51f49135a9bf8b72bea4efed3bdb2b1384502882277: Status 404 returned error can't find the container with id 5e10096c44c423cf3557a51f49135a9bf8b72bea4efed3bdb2b1384502882277
Sep 11 12:02:49.630744 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:49.630711    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-12d2a9-57b894675b-phbq4" event={"ID":"2418f3a0-c6e5-4a97-b33d-f5f62fd70741","Type":"ContainerStarted","Data":"5e10096c44c423cf3557a51f49135a9bf8b72bea4efed3bdb2b1384502882277"}
Sep 11 12:02:50.229035 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:50.228998    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:50.638331 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:50.638292    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-12d2a9-57b894675b-phbq4" event={"ID":"2418f3a0-c6e5-4a97-b33d-f5f62fd70741","Type":"ContainerStarted","Data":"67a38218a967691f68d60bcbbb97fabdad8e6a366c1b748208745d28c8237d6e"}
Sep 11 12:02:50.638546 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:50.638442    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-12d2a9-57b894675b-phbq4"
Sep 11 12:02:50.678998 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:50.678926    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-12d2a9-57b894675b-phbq4" podStartSLOduration=1.530051323 podStartE2EDuration="2.678897535s" totalImagesPullingTime="1.148846212s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:02:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:02:50.675606079 +0000 UTC m=+874.573047825" watchObservedRunningTime="2026-09-11 12:02:50.678897535 +0000 UTC m=+874.576339283"
Sep 11 12:02:50.976952 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:50.976913    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:52.613945 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:52.613895    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:53.709526 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:53.709489    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:56.467691 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:56.467641    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:56.649928 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:56.649893    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-12d2a9-57b894675b-phbq4"
Sep 11 12:02:59.473532 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:02:59.473498    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:04.577849 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:04.577814    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:03:05.492478 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:05.492444    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:06.262145 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:06.262046    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:06.367539 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:06.367507    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:06.474718 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:06.474679    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:06.736770 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:06.736728    2579 generic.go:356] "Generic (PLEG): container finished" podID="36e69a19-0f39-41aa-b80e-e60b2b9cf3c7" containerID="e5491aa364bf970cb3d5d9121e4da22b60df276ed7520131ef95cbeceb42b291" exitCode=0
Sep 11 12:03:06.736770 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:06.736758    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv" event={"ID":"36e69a19-0f39-41aa-b80e-e60b2b9cf3c7","Type":"ContainerDied","Data":"e5491aa364bf970cb3d5d9121e4da22b60df276ed7520131ef95cbeceb42b291"}
Sep 11 12:03:06.737288 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:06.737259    2579 scope.go:117] "RemoveContainer" containerID="e5491aa364bf970cb3d5d9121e4da22b60df276ed7520131ef95cbeceb42b291"
Sep 11 12:03:06.738567 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:06.738533    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 12:03:06.738692 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:06.738620    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 12:03:06.738692 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:06.738673    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 12:03:07.681944 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:07.681910    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:07.744892 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:07.744861    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv" event={"ID":"36e69a19-0f39-41aa-b80e-e60b2b9cf3c7","Type":"ContainerStarted","Data":"c6587bc5c46ed78642d9cf1395f52b68f153af5e25cfeb65a33d1cfe68d081cf"}
Sep 11 12:03:07.823042 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:07.823010    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:07.870511 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:07.870478    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs"]
Sep 11 12:03:07.873107 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:07.873090    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs"
Sep 11 12:03:07.879531 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:07.879464    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-150d6af2-gw-openshift-default-dockercfg-pf6rh\""
Sep 11 12:03:07.893299 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:07.893269    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs"]
Sep 11 12:03:07.904581 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:07.904538    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bq8m6\" (UniqueName: \"kubernetes.io/projected/e4138f6a-1f19-4f03-8177-d417cecfaba5-kube-api-access-bq8m6\") pod \"e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs\" (UID: \"e4138f6a-1f19-4f03-8177-d417cecfaba5\") " pod="openshift-ingress/e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs"
Sep 11 12:03:07.904762 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:07.904600    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4138f6a-1f19-4f03-8177-d417cecfaba5-istio-data\") pod \"e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs\" (UID: \"e4138f6a-1f19-4f03-8177-d417cecfaba5\") " pod="openshift-ingress/e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs"
Sep 11 12:03:07.904762 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:07.904636    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4138f6a-1f19-4f03-8177-d417cecfaba5-workload-certs\") pod \"e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs\" (UID: \"e4138f6a-1f19-4f03-8177-d417cecfaba5\") " pod="openshift-ingress/e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs"
Sep 11 12:03:07.904762 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:07.904739    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4138f6a-1f19-4f03-8177-d417cecfaba5-workload-socket\") pod \"e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs\" (UID: \"e4138f6a-1f19-4f03-8177-d417cecfaba5\") " pod="openshift-ingress/e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs"
Sep 11 12:03:07.904949 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:07.904765    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4138f6a-1f19-4f03-8177-d417cecfaba5-istio-podinfo\") pod \"e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs\" (UID: \"e4138f6a-1f19-4f03-8177-d417cecfaba5\") " pod="openshift-ingress/e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs"
Sep 11 12:03:07.904949 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:07.904801    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4138f6a-1f19-4f03-8177-d417cecfaba5-credential-socket\") pod \"e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs\" (UID: \"e4138f6a-1f19-4f03-8177-d417cecfaba5\") " pod="openshift-ingress/e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs"
Sep 11 12:03:07.904949 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:07.904825    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4138f6a-1f19-4f03-8177-d417cecfaba5-istiod-ca-cert\") pod \"e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs\" (UID: \"e4138f6a-1f19-4f03-8177-d417cecfaba5\") " pod="openshift-ingress/e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs"
Sep 11 12:03:07.904949 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:07.904859    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4138f6a-1f19-4f03-8177-d417cecfaba5-istio-envoy\") pod \"e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs\" (UID: \"e4138f6a-1f19-4f03-8177-d417cecfaba5\") " pod="openshift-ingress/e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs"
Sep 11 12:03:07.904949 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:07.904882    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4138f6a-1f19-4f03-8177-d417cecfaba5-istio-token\") pod \"e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs\" (UID: \"e4138f6a-1f19-4f03-8177-d417cecfaba5\") " pod="openshift-ingress/e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs"
Sep 11 12:03:07.945458 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:07.945323    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:07.963673 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:07.963628    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv"
Sep 11 12:03:07.964017 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:07.963814    2579 patch_prober.go:28] interesting pod/e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Sep 11 12:03:07.964017 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:07.963885    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv" podUID="36e69a19-0f39-41aa-b80e-e60b2b9cf3c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Sep 11 12:03:08.005622 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:08.005570    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4138f6a-1f19-4f03-8177-d417cecfaba5-workload-socket\") pod \"e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs\" (UID: \"e4138f6a-1f19-4f03-8177-d417cecfaba5\") " pod="openshift-ingress/e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs"
Sep 11 12:03:08.005936 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:08.005913    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4138f6a-1f19-4f03-8177-d417cecfaba5-istio-podinfo\") pod \"e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs\" (UID: \"e4138f6a-1f19-4f03-8177-d417cecfaba5\") " pod="openshift-ingress/e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs"
Sep 11 12:03:08.006105 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:08.006088    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4138f6a-1f19-4f03-8177-d417cecfaba5-credential-socket\") pod \"e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs\" (UID: \"e4138f6a-1f19-4f03-8177-d417cecfaba5\") " pod="openshift-ingress/e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs"
Sep 11 12:03:08.006217 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:08.006202    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4138f6a-1f19-4f03-8177-d417cecfaba5-istiod-ca-cert\") pod \"e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs\" (UID: \"e4138f6a-1f19-4f03-8177-d417cecfaba5\") " pod="openshift-ingress/e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs"
Sep 11 12:03:08.006351 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:08.006336    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4138f6a-1f19-4f03-8177-d417cecfaba5-istio-envoy\") pod \"e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs\" (UID: \"e4138f6a-1f19-4f03-8177-d417cecfaba5\") " pod="openshift-ingress/e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs"
Sep 11 12:03:08.006491 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:08.006474    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4138f6a-1f19-4f03-8177-d417cecfaba5-istio-token\") pod \"e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs\" (UID: \"e4138f6a-1f19-4f03-8177-d417cecfaba5\") " pod="openshift-ingress/e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs"
Sep 11 12:03:08.006685 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:08.006665    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bq8m6\" (UniqueName: \"kubernetes.io/projected/e4138f6a-1f19-4f03-8177-d417cecfaba5-kube-api-access-bq8m6\") pod \"e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs\" (UID: \"e4138f6a-1f19-4f03-8177-d417cecfaba5\") " pod="openshift-ingress/e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs"
Sep 11 12:03:08.006819 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:08.006801    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4138f6a-1f19-4f03-8177-d417cecfaba5-istio-data\") pod \"e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs\" (UID: \"e4138f6a-1f19-4f03-8177-d417cecfaba5\") " pod="openshift-ingress/e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs"
Sep 11 12:03:08.006939 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:08.006923    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4138f6a-1f19-4f03-8177-d417cecfaba5-workload-certs\") pod \"e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs\" (UID: \"e4138f6a-1f19-4f03-8177-d417cecfaba5\") " pod="openshift-ingress/e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs"
Sep 11 12:03:08.007428 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:08.007066    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4138f6a-1f19-4f03-8177-d417cecfaba5-istiod-ca-cert\") pod \"e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs\" (UID: \"e4138f6a-1f19-4f03-8177-d417cecfaba5\") " pod="openshift-ingress/e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs"
Sep 11 12:03:08.007428 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:08.006160    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4138f6a-1f19-4f03-8177-d417cecfaba5-workload-socket\") pod \"e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs\" (UID: \"e4138f6a-1f19-4f03-8177-d417cecfaba5\") " pod="openshift-ingress/e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs"
Sep 11 12:03:08.007428 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:08.006415    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4138f6a-1f19-4f03-8177-d417cecfaba5-credential-socket\") pod \"e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs\" (UID: \"e4138f6a-1f19-4f03-8177-d417cecfaba5\") " pod="openshift-ingress/e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs"
Sep 11 12:03:08.007428 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:08.007359    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4138f6a-1f19-4f03-8177-d417cecfaba5-workload-certs\") pod \"e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs\" (UID: \"e4138f6a-1f19-4f03-8177-d417cecfaba5\") " pod="openshift-ingress/e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs"
Sep 11 12:03:08.007428 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:08.007380    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4138f6a-1f19-4f03-8177-d417cecfaba5-istio-data\") pod \"e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs\" (UID: \"e4138f6a-1f19-4f03-8177-d417cecfaba5\") " pod="openshift-ingress/e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs"
Sep 11 12:03:08.008977 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:08.008949    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4138f6a-1f19-4f03-8177-d417cecfaba5-istio-envoy\") pod \"e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs\" (UID: \"e4138f6a-1f19-4f03-8177-d417cecfaba5\") " pod="openshift-ingress/e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs"
Sep 11 12:03:08.009288 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:08.009267    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4138f6a-1f19-4f03-8177-d417cecfaba5-istio-podinfo\") pod \"e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs\" (UID: \"e4138f6a-1f19-4f03-8177-d417cecfaba5\") " pod="openshift-ingress/e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs"
Sep 11 12:03:08.019087 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:08.019029    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4138f6a-1f19-4f03-8177-d417cecfaba5-istio-token\") pod \"e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs\" (UID: \"e4138f6a-1f19-4f03-8177-d417cecfaba5\") " pod="openshift-ingress/e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs"
Sep 11 12:03:08.021966 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:08.021938    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bq8m6\" (UniqueName: \"kubernetes.io/projected/e4138f6a-1f19-4f03-8177-d417cecfaba5-kube-api-access-bq8m6\") pod \"e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs\" (UID: \"e4138f6a-1f19-4f03-8177-d417cecfaba5\") " pod="openshift-ingress/e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs"
Sep 11 12:03:08.185560 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:08.185516    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs"
Sep 11 12:03:08.297414 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:08.296767    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:08.488022 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:08.487895    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs"]
Sep 11 12:03:08.498141 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:08.498093    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 12:03:08.498271 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:08.498200    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 12:03:08.498271 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:08.498248    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 12:03:08.754315 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:08.754212    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs" event={"ID":"e4138f6a-1f19-4f03-8177-d417cecfaba5","Type":"ContainerStarted","Data":"dc30ec3dfdac40c1b2e64f6737c020e8b40a68bbfe79c2d60ab38a41a63873bf"}
Sep 11 12:03:08.754315 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:08.754282    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs" event={"ID":"e4138f6a-1f19-4f03-8177-d417cecfaba5","Type":"ContainerStarted","Data":"b1283383e8bce18cd8258274d4b65ae99f9bc3e734513700e8798f86ef539be2"}
Sep 11 12:03:08.964018 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:08.963983    2579 patch_prober.go:28] interesting pod/e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Sep 11 12:03:08.964191 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:08.964052    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv" podUID="36e69a19-0f39-41aa-b80e-e60b2b9cf3c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Sep 11 12:03:09.187138 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:09.186756    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs"
Sep 11 12:03:09.198621 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:09.198586    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs"
Sep 11 12:03:09.239470 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:09.239385    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs" podStartSLOduration=2.239367661 podStartE2EDuration="2.239367661s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:03:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:03:08.812117673 +0000 UTC m=+892.709559419" watchObservedRunningTime="2026-09-11 12:03:09.239367661 +0000 UTC m=+893.136809452"
Sep 11 12:03:09.760086 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:09.760052    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs"
Sep 11 12:03:09.761051 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:09.761032    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs"
Sep 11 12:03:09.780369 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:09.780330    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:09.963886 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:09.963843    2579 patch_prober.go:28] interesting pod/e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Sep 11 12:03:09.964072 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:09.963915    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv" podUID="36e69a19-0f39-41aa-b80e-e60b2b9cf3c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Sep 11 12:03:10.387767 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:10.387718    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:10.963373 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:10.963331    2579 patch_prober.go:28] interesting pod/e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Sep 11 12:03:10.963766 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:10.963443    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv" podUID="36e69a19-0f39-41aa-b80e-e60b2b9cf3c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Sep 11 12:03:11.519739 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:11.519701    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:11.634054 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:11.634018    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:11.741468 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:11.741428    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-150d6af2-7f5946564b-9j7mr"]
Sep 11 12:03:11.744276 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:11.744259    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-150d6af2-7f5946564b-9j7mr"
Sep 11 12:03:11.748345 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:11.748321    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-150d6af2-dockercfg-b5wxh\""
Sep 11 12:03:11.748345 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:11.748338    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-150d6af2\""
Sep 11 12:03:11.763945 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:11.763914    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-150d6af2-7f5946564b-9j7mr"]
Sep 11 12:03:11.845854 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:11.845826    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4d2m2\" (UniqueName: \"kubernetes.io/projected/f4d53ce7-e017-436f-b2fe-848b4cdea538-kube-api-access-4d2m2\") pod \"payload-pre-processing-e2e-migrate-150d6af2-7f5946564b-9j7mr\" (UID: \"f4d53ce7-e017-436f-b2fe-848b4cdea538\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-150d6af2-7f5946564b-9j7mr"
Sep 11 12:03:11.846065 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:11.846031    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4d53ce7-e017-436f-b2fe-848b4cdea538-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-150d6af2-7f5946564b-9j7mr\" (UID: \"f4d53ce7-e017-436f-b2fe-848b4cdea538\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-150d6af2-7f5946564b-9j7mr"
Sep 11 12:03:11.847112 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:11.847083    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-150d6af2-9d78f58cd-r4lqf"]
Sep 11 12:03:11.850164 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:11.850142    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-150d6af2-9d78f58cd-r4lqf"
Sep 11 12:03:11.861609 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:11.861585    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-150d6af2-9d78f58cd-r4lqf"]
Sep 11 12:03:11.947248 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:11.947206    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0e7c9c83-409d-4e70-8baf-dffef0609e88-plugins-config-volume\") pod \"payload-processing-e2e-migrate-150d6af2-9d78f58cd-r4lqf\" (UID: \"0e7c9c83-409d-4e70-8baf-dffef0609e88\") " pod="openshift-ingress/payload-processing-e2e-migrate-150d6af2-9d78f58cd-r4lqf"
Sep 11 12:03:11.947460 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:11.947261    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4d53ce7-e017-436f-b2fe-848b4cdea538-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-150d6af2-7f5946564b-9j7mr\" (UID: \"f4d53ce7-e017-436f-b2fe-848b4cdea538\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-150d6af2-7f5946564b-9j7mr"
Sep 11 12:03:11.947460 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:11.947335    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkl5n\" (UniqueName: \"kubernetes.io/projected/0e7c9c83-409d-4e70-8baf-dffef0609e88-kube-api-access-gkl5n\") pod \"payload-processing-e2e-migrate-150d6af2-9d78f58cd-r4lqf\" (UID: \"0e7c9c83-409d-4e70-8baf-dffef0609e88\") " pod="openshift-ingress/payload-processing-e2e-migrate-150d6af2-9d78f58cd-r4lqf"
Sep 11 12:03:11.947579 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:11.947467    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4d2m2\" (UniqueName: \"kubernetes.io/projected/f4d53ce7-e017-436f-b2fe-848b4cdea538-kube-api-access-4d2m2\") pod \"payload-pre-processing-e2e-migrate-150d6af2-7f5946564b-9j7mr\" (UID: \"f4d53ce7-e017-436f-b2fe-848b4cdea538\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-150d6af2-7f5946564b-9j7mr"
Sep 11 12:03:11.947928 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:11.947909    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4d53ce7-e017-436f-b2fe-848b4cdea538-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-150d6af2-7f5946564b-9j7mr\" (UID: \"f4d53ce7-e017-436f-b2fe-848b4cdea538\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-150d6af2-7f5946564b-9j7mr"
Sep 11 12:03:11.957467 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:11.957442    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4d2m2\" (UniqueName: \"kubernetes.io/projected/f4d53ce7-e017-436f-b2fe-848b4cdea538-kube-api-access-4d2m2\") pod \"payload-pre-processing-e2e-migrate-150d6af2-7f5946564b-9j7mr\" (UID: \"f4d53ce7-e017-436f-b2fe-848b4cdea538\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-150d6af2-7f5946564b-9j7mr"
Sep 11 12:03:11.963457 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:11.963426    2579 patch_prober.go:28] interesting pod/e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Sep 11 12:03:11.963831 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:11.963478    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv" podUID="36e69a19-0f39-41aa-b80e-e60b2b9cf3c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Sep 11 12:03:12.048386 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:12.048346    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0e7c9c83-409d-4e70-8baf-dffef0609e88-plugins-config-volume\") pod \"payload-processing-e2e-migrate-150d6af2-9d78f58cd-r4lqf\" (UID: \"0e7c9c83-409d-4e70-8baf-dffef0609e88\") " pod="openshift-ingress/payload-processing-e2e-migrate-150d6af2-9d78f58cd-r4lqf"
Sep 11 12:03:12.048575 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:12.048437    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkl5n\" (UniqueName: \"kubernetes.io/projected/0e7c9c83-409d-4e70-8baf-dffef0609e88-kube-api-access-gkl5n\") pod \"payload-processing-e2e-migrate-150d6af2-9d78f58cd-r4lqf\" (UID: \"0e7c9c83-409d-4e70-8baf-dffef0609e88\") " pod="openshift-ingress/payload-processing-e2e-migrate-150d6af2-9d78f58cd-r4lqf"
Sep 11 12:03:12.049061 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:12.049041    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0e7c9c83-409d-4e70-8baf-dffef0609e88-plugins-config-volume\") pod \"payload-processing-e2e-migrate-150d6af2-9d78f58cd-r4lqf\" (UID: \"0e7c9c83-409d-4e70-8baf-dffef0609e88\") " pod="openshift-ingress/payload-processing-e2e-migrate-150d6af2-9d78f58cd-r4lqf"
Sep 11 12:03:12.055213 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:12.055185    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-150d6af2-7f5946564b-9j7mr"
Sep 11 12:03:12.059160 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:12.059139    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkl5n\" (UniqueName: \"kubernetes.io/projected/0e7c9c83-409d-4e70-8baf-dffef0609e88-kube-api-access-gkl5n\") pod \"payload-processing-e2e-migrate-150d6af2-9d78f58cd-r4lqf\" (UID: \"0e7c9c83-409d-4e70-8baf-dffef0609e88\") " pod="openshift-ingress/payload-processing-e2e-migrate-150d6af2-9d78f58cd-r4lqf"
Sep 11 12:03:12.161806 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:12.161772    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-150d6af2-9d78f58cd-r4lqf"
Sep 11 12:03:12.204416 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:12.199897    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:12.370271 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:12.370240    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-150d6af2-7f5946564b-9j7mr"]
Sep 11 12:03:12.370577 ip-10-0-142-145 kubenswrapper[2579]: W0911 12:03:12.370553    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4d53ce7_e017_436f_b2fe_848b4cdea538.slice/crio-c5f8709cd559a6a912a8ae955e14e6485c0506bbf91af7d609f5566427e3dda3 WatchSource:0}: Error finding container c5f8709cd559a6a912a8ae955e14e6485c0506bbf91af7d609f5566427e3dda3: Status 404 returned error can't find the container with id c5f8709cd559a6a912a8ae955e14e6485c0506bbf91af7d609f5566427e3dda3
Sep 11 12:03:12.484833 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:12.484767    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-150d6af2-9d78f58cd-r4lqf"]
Sep 11 12:03:12.489325 ip-10-0-142-145 kubenswrapper[2579]: W0911 12:03:12.489281    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e7c9c83_409d_4e70_8baf_dffef0609e88.slice/crio-8121f226937eb0554d85b8d8ff3576a7e81dbc91f20bfb19e68a70358386bf98 WatchSource:0}: Error finding container 8121f226937eb0554d85b8d8ff3576a7e81dbc91f20bfb19e68a70358386bf98: Status 404 returned error can't find the container with id 8121f226937eb0554d85b8d8ff3576a7e81dbc91f20bfb19e68a70358386bf98
Sep 11 12:03:12.780313 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:12.779680    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-150d6af2-7f5946564b-9j7mr" event={"ID":"f4d53ce7-e017-436f-b2fe-848b4cdea538","Type":"ContainerStarted","Data":"3298b243b9124f21bff9400fd3faadb8ca37b289cebfb72bf6a0a5467c83f1dd"}
Sep 11 12:03:12.780313 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:12.779745    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-150d6af2-7f5946564b-9j7mr" event={"ID":"f4d53ce7-e017-436f-b2fe-848b4cdea538","Type":"ContainerStarted","Data":"c5f8709cd559a6a912a8ae955e14e6485c0506bbf91af7d609f5566427e3dda3"}
Sep 11 12:03:12.780313 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:12.779805    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-150d6af2-7f5946564b-9j7mr"
Sep 11 12:03:12.782065 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:12.782024    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-150d6af2-9d78f58cd-r4lqf" event={"ID":"0e7c9c83-409d-4e70-8baf-dffef0609e88","Type":"ContainerStarted","Data":"e545bfa29171d816a8d507635fd233de702a0ec8274a561fa4834af2b74204ea"}
Sep 11 12:03:12.782188 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:12.782091    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-150d6af2-9d78f58cd-r4lqf" event={"ID":"0e7c9c83-409d-4e70-8baf-dffef0609e88","Type":"ContainerStarted","Data":"8121f226937eb0554d85b8d8ff3576a7e81dbc91f20bfb19e68a70358386bf98"}
Sep 11 12:03:12.782731 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:12.782707    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-150d6af2-9d78f58cd-r4lqf"
Sep 11 12:03:12.802105 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:12.802037    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-150d6af2-7f5946564b-9j7mr" podStartSLOduration=1.8020145749999998 podStartE2EDuration="1.802014575s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:03:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:03:12.799529184 +0000 UTC m=+896.696970930" watchObservedRunningTime="2026-09-11 12:03:12.802014575 +0000 UTC m=+896.699456323"
Sep 11 12:03:12.823245 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:12.823186    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-150d6af2-9d78f58cd-r4lqf" podStartSLOduration=1.823166353 podStartE2EDuration="1.823166353s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:03:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:03:12.820106738 +0000 UTC m=+896.717548484" watchObservedRunningTime="2026-09-11 12:03:12.823166353 +0000 UTC m=+896.720608135"
Sep 11 12:03:12.963406 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:12.963360    2579 patch_prober.go:28] interesting pod/e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Sep 11 12:03:12.963556 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:12.963437    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv" podUID="36e69a19-0f39-41aa-b80e-e60b2b9cf3c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Sep 11 12:03:13.206423 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:13.206250    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:13.968118 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:13.968089    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv"
Sep 11 12:03:13.968580 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:13.968499    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv"
Sep 11 12:03:13.969193 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:13.969172    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv"
Sep 11 12:03:16.517266 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:16.517233    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-ch96n_1f0f9fa5-cbd1-468b-ae81-82b48c101d37/console-operator/2.log"
Sep 11 12:03:16.518787 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:16.518765    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-ch96n_1f0f9fa5-cbd1-468b-ae81-82b48c101d37/console-operator/2.log"
Sep 11 12:03:23.792468 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:23.792429    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-150d6af2-7f5946564b-9j7mr"
Sep 11 12:03:23.874203 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:23.874158    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:03:24.797037 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:24.797002    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-150d6af2-9d78f58cd-r4lqf"
Sep 11 12:03:25.217246 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:25.216942    2579 patch_prober.go:28] interesting pod/e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": read tcp 10.134.0.2:45022->10.134.0.41:15021: read: connection reset by peer" start-of-body=
Sep 11 12:03:25.217246 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:25.217017    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs" podUID="e4138f6a-1f19-4f03-8177-d417cecfaba5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": read tcp 10.134.0.2:45022->10.134.0.41:15021: read: connection reset by peer"
Sep 11 12:03:25.859097 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:25.859060    2579 generic.go:356] "Generic (PLEG): container finished" podID="e4138f6a-1f19-4f03-8177-d417cecfaba5" containerID="dc30ec3dfdac40c1b2e64f6737c020e8b40a68bbfe79c2d60ab38a41a63873bf" exitCode=0
Sep 11 12:03:25.859536 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:25.859133    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs" event={"ID":"e4138f6a-1f19-4f03-8177-d417cecfaba5","Type":"ContainerDied","Data":"dc30ec3dfdac40c1b2e64f6737c020e8b40a68bbfe79c2d60ab38a41a63873bf"}
Sep 11 12:03:25.859583 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:25.859556    2579 scope.go:117] "RemoveContainer" containerID="dc30ec3dfdac40c1b2e64f6737c020e8b40a68bbfe79c2d60ab38a41a63873bf"
Sep 11 12:03:25.860698 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:25.860670    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 12:03:25.860823 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:25.860754    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 12:03:25.860823 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:25.860800    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 12:03:26.866937 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:26.866904    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs" event={"ID":"e4138f6a-1f19-4f03-8177-d417cecfaba5","Type":"ContainerStarted","Data":"054f579f72d5abb9ce40005d563cbf17efe00890df237142281879d56f31aa70"}
Sep 11 12:03:27.187234 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:27.187126    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs"
Sep 11 12:03:27.187591 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:27.187551    2579 patch_prober.go:28] interesting pod/e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Sep 11 12:03:27.187800 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:27.187773    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs" podUID="e4138f6a-1f19-4f03-8177-d417cecfaba5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Sep 11 12:03:28.187081 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:28.187045    2579 patch_prober.go:28] interesting pod/e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Sep 11 12:03:28.187793 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:28.187759    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs" podUID="e4138f6a-1f19-4f03-8177-d417cecfaba5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Sep 11 12:03:29.187055 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:29.187006    2579 patch_prober.go:28] interesting pod/e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Sep 11 12:03:29.187274 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:29.187078    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs" podUID="e4138f6a-1f19-4f03-8177-d417cecfaba5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Sep 11 12:03:30.186870 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:30.186830    2579 patch_prober.go:28] interesting pod/e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Sep 11 12:03:30.187169 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:30.187136    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs" podUID="e4138f6a-1f19-4f03-8177-d417cecfaba5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Sep 11 12:03:31.187384 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:31.187345    2579 patch_prober.go:28] interesting pod/e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Sep 11 12:03:31.187978 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:31.187947    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs" podUID="e4138f6a-1f19-4f03-8177-d417cecfaba5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Sep 11 12:03:32.189590 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:32.189559    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs"
Sep 11 12:03:32.189996 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:32.189978    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs"
Sep 11 12:03:32.190596 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:32.190579    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs"
Sep 11 12:03:36.169908 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:36.169867    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:42.289739 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:42.289701    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:42.332175 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:42.332106    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-150d6af2-9d78f58cd-r4lqf"]
Sep 11 12:03:42.332584 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:42.332431    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-150d6af2-9d78f58cd-r4lqf" podUID="0e7c9c83-409d-4e70-8baf-dffef0609e88" containerName="payload-processing" containerID="cri-o://e545bfa29171d816a8d507635fd233de702a0ec8274a561fa4834af2b74204ea" gracePeriod=30
Sep 11 12:03:42.344230 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:42.344202    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-150d6af2-7f5946564b-9j7mr"]
Sep 11 12:03:42.344630 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:42.344567    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-150d6af2-7f5946564b-9j7mr" podUID="f4d53ce7-e017-436f-b2fe-848b4cdea538" containerName="payload-pre-processing" containerID="cri-o://3298b243b9124f21bff9400fd3faadb8ca37b289cebfb72bf6a0a5467c83f1dd" gracePeriod=30
Sep 11 12:03:42.375973 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:42.375944    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:42.639955 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:42.639922    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-150d6af2-9d78f58cd-r4lqf"
Sep 11 12:03:42.642925 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:42.642904    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-150d6af2-7f5946564b-9j7mr"
Sep 11 12:03:42.693774 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:42.693739    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs"]
Sep 11 12:03:42.694249 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:42.694211    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs" podUID="e4138f6a-1f19-4f03-8177-d417cecfaba5" containerName="istio-proxy" containerID="cri-o://054f579f72d5abb9ce40005d563cbf17efe00890df237142281879d56f31aa70" gracePeriod=30
Sep 11 12:03:42.756927 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:42.756887    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0e7c9c83-409d-4e70-8baf-dffef0609e88-plugins-config-volume\") pod \"0e7c9c83-409d-4e70-8baf-dffef0609e88\" (UID: \"0e7c9c83-409d-4e70-8baf-dffef0609e88\") "
Sep 11 12:03:42.757130 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:42.756956    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4d53ce7-e017-436f-b2fe-848b4cdea538-plugins-config-volume\") pod \"f4d53ce7-e017-436f-b2fe-848b4cdea538\" (UID: \"f4d53ce7-e017-436f-b2fe-848b4cdea538\") "
Sep 11 12:03:42.757130 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:42.757001    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4d2m2\" (UniqueName: \"kubernetes.io/projected/f4d53ce7-e017-436f-b2fe-848b4cdea538-kube-api-access-4d2m2\") pod \"f4d53ce7-e017-436f-b2fe-848b4cdea538\" (UID: \"f4d53ce7-e017-436f-b2fe-848b4cdea538\") "
Sep 11 12:03:42.757130 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:42.757047    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gkl5n\" (UniqueName: \"kubernetes.io/projected/0e7c9c83-409d-4e70-8baf-dffef0609e88-kube-api-access-gkl5n\") pod \"0e7c9c83-409d-4e70-8baf-dffef0609e88\" (UID: \"0e7c9c83-409d-4e70-8baf-dffef0609e88\") "
Sep 11 12:03:42.757413 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:42.757371    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e7c9c83-409d-4e70-8baf-dffef0609e88-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0e7c9c83-409d-4e70-8baf-dffef0609e88" (UID: "0e7c9c83-409d-4e70-8baf-dffef0609e88"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:03:42.757497 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:42.757377    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4d53ce7-e017-436f-b2fe-848b4cdea538-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f4d53ce7-e017-436f-b2fe-848b4cdea538" (UID: "f4d53ce7-e017-436f-b2fe-848b4cdea538"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:03:42.759292 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:42.759269    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4d53ce7-e017-436f-b2fe-848b4cdea538-kube-api-access-4d2m2" (OuterVolumeSpecName: "kube-api-access-4d2m2") pod "f4d53ce7-e017-436f-b2fe-848b4cdea538" (UID: "f4d53ce7-e017-436f-b2fe-848b4cdea538"). InnerVolumeSpecName "kube-api-access-4d2m2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:03:42.759443 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:42.759317    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e7c9c83-409d-4e70-8baf-dffef0609e88-kube-api-access-gkl5n" (OuterVolumeSpecName: "kube-api-access-gkl5n") pod "0e7c9c83-409d-4e70-8baf-dffef0609e88" (UID: "0e7c9c83-409d-4e70-8baf-dffef0609e88"). InnerVolumeSpecName "kube-api-access-gkl5n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:03:42.858308 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:42.858277    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4d2m2\" (UniqueName: \"kubernetes.io/projected/f4d53ce7-e017-436f-b2fe-848b4cdea538-kube-api-access-4d2m2\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:03:42.858308 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:42.858309    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gkl5n\" (UniqueName: \"kubernetes.io/projected/0e7c9c83-409d-4e70-8baf-dffef0609e88-kube-api-access-gkl5n\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:03:42.858539 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:42.858318    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0e7c9c83-409d-4e70-8baf-dffef0609e88-plugins-config-volume\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:03:42.858539 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:42.858327    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4d53ce7-e017-436f-b2fe-848b4cdea538-plugins-config-volume\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:03:42.881547 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:42.881483    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:42.967611 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:42.967578    2579 generic.go:356] "Generic (PLEG): container finished" podID="f4d53ce7-e017-436f-b2fe-848b4cdea538" containerID="3298b243b9124f21bff9400fd3faadb8ca37b289cebfb72bf6a0a5467c83f1dd" exitCode=0
Sep 11 12:03:42.967779 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:42.967626    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-150d6af2-7f5946564b-9j7mr"
Sep 11 12:03:42.967779 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:42.967663    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-150d6af2-7f5946564b-9j7mr" event={"ID":"f4d53ce7-e017-436f-b2fe-848b4cdea538","Type":"ContainerDied","Data":"3298b243b9124f21bff9400fd3faadb8ca37b289cebfb72bf6a0a5467c83f1dd"}
Sep 11 12:03:42.967779 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:42.967726    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-150d6af2-7f5946564b-9j7mr" event={"ID":"f4d53ce7-e017-436f-b2fe-848b4cdea538","Type":"ContainerDied","Data":"c5f8709cd559a6a912a8ae955e14e6485c0506bbf91af7d609f5566427e3dda3"}
Sep 11 12:03:42.967779 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:42.967760    2579 scope.go:117] "RemoveContainer" containerID="3298b243b9124f21bff9400fd3faadb8ca37b289cebfb72bf6a0a5467c83f1dd"
Sep 11 12:03:42.969109 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:42.969091    2579 generic.go:356] "Generic (PLEG): container finished" podID="0e7c9c83-409d-4e70-8baf-dffef0609e88" containerID="e545bfa29171d816a8d507635fd233de702a0ec8274a561fa4834af2b74204ea" exitCode=0
Sep 11 12:03:42.969206 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:42.969136    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-150d6af2-9d78f58cd-r4lqf"
Sep 11 12:03:42.969206 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:42.969185    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-150d6af2-9d78f58cd-r4lqf" event={"ID":"0e7c9c83-409d-4e70-8baf-dffef0609e88","Type":"ContainerDied","Data":"e545bfa29171d816a8d507635fd233de702a0ec8274a561fa4834af2b74204ea"}
Sep 11 12:03:42.969290 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:42.969239    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-150d6af2-9d78f58cd-r4lqf" event={"ID":"0e7c9c83-409d-4e70-8baf-dffef0609e88","Type":"ContainerDied","Data":"8121f226937eb0554d85b8d8ff3576a7e81dbc91f20bfb19e68a70358386bf98"}
Sep 11 12:03:42.979491 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:42.979476    2579 scope.go:117] "RemoveContainer" containerID="3298b243b9124f21bff9400fd3faadb8ca37b289cebfb72bf6a0a5467c83f1dd"
Sep 11 12:03:42.979844 ip-10-0-142-145 kubenswrapper[2579]: E0911 12:03:42.979815    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3298b243b9124f21bff9400fd3faadb8ca37b289cebfb72bf6a0a5467c83f1dd\": container with ID starting with 3298b243b9124f21bff9400fd3faadb8ca37b289cebfb72bf6a0a5467c83f1dd not found: ID does not exist" containerID="3298b243b9124f21bff9400fd3faadb8ca37b289cebfb72bf6a0a5467c83f1dd"
Sep 11 12:03:42.979919 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:42.979859    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3298b243b9124f21bff9400fd3faadb8ca37b289cebfb72bf6a0a5467c83f1dd"} err="failed to get container status \"3298b243b9124f21bff9400fd3faadb8ca37b289cebfb72bf6a0a5467c83f1dd\": rpc error: code = NotFound desc = could not find container \"3298b243b9124f21bff9400fd3faadb8ca37b289cebfb72bf6a0a5467c83f1dd\": container with ID starting with 3298b243b9124f21bff9400fd3faadb8ca37b289cebfb72bf6a0a5467c83f1dd not found: ID does not exist"
Sep 11 12:03:42.979919 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:42.979882    2579 scope.go:117] "RemoveContainer" containerID="e545bfa29171d816a8d507635fd233de702a0ec8274a561fa4834af2b74204ea"
Sep 11 12:03:42.988950 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:42.988929    2579 scope.go:117] "RemoveContainer" containerID="e545bfa29171d816a8d507635fd233de702a0ec8274a561fa4834af2b74204ea"
Sep 11 12:03:42.989270 ip-10-0-142-145 kubenswrapper[2579]: E0911 12:03:42.989248    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e545bfa29171d816a8d507635fd233de702a0ec8274a561fa4834af2b74204ea\": container with ID starting with e545bfa29171d816a8d507635fd233de702a0ec8274a561fa4834af2b74204ea not found: ID does not exist" containerID="e545bfa29171d816a8d507635fd233de702a0ec8274a561fa4834af2b74204ea"
Sep 11 12:03:42.989323 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:42.989278    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e545bfa29171d816a8d507635fd233de702a0ec8274a561fa4834af2b74204ea"} err="failed to get container status \"e545bfa29171d816a8d507635fd233de702a0ec8274a561fa4834af2b74204ea\": rpc error: code = NotFound desc = could not find container \"e545bfa29171d816a8d507635fd233de702a0ec8274a561fa4834af2b74204ea\": container with ID starting with e545bfa29171d816a8d507635fd233de702a0ec8274a561fa4834af2b74204ea not found: ID does not exist"
Sep 11 12:03:42.998071 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:42.998047    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-150d6af2-7f5946564b-9j7mr"]
Sep 11 12:03:43.002352 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:43.002329    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:43.003137 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:43.003116    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-150d6af2-7f5946564b-9j7mr"]
Sep 11 12:03:43.015994 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:43.015971    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-150d6af2-9d78f58cd-r4lqf"]
Sep 11 12:03:43.019825 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:43.019804    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-150d6af2-9d78f58cd-r4lqf"]
Sep 11 12:03:43.189944 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:43.189851    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:43.552497 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:43.552375    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:43.730788 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:43.730754    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:44.577379 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:44.577345    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e7c9c83-409d-4e70-8baf-dffef0609e88" path="/var/lib/kubelet/pods/0e7c9c83-409d-4e70-8baf-dffef0609e88/volumes"
Sep 11 12:03:44.577841 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:44.577823    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4d53ce7-e017-436f-b2fe-848b4cdea538" path="/var/lib/kubelet/pods/f4d53ce7-e017-436f-b2fe-848b4cdea538/volumes"
Sep 11 12:03:46.075076 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:46.075039    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:46.187271 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:46.187232    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:47.192365 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:47.192322    2579 patch_prober.go:28] interesting pod/e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
Sep 11 12:03:47.192757 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:47.192384    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs" podUID="e4138f6a-1f19-4f03-8177-d417cecfaba5" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 12:03:47.478039 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:47.477937    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:47.609686 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:47.609640    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:47.735156 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:47.735073    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:47.984578 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:47.984554    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs"
Sep 11 12:03:48.010248 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.010166    2579 generic.go:356] "Generic (PLEG): container finished" podID="e4138f6a-1f19-4f03-8177-d417cecfaba5" containerID="054f579f72d5abb9ce40005d563cbf17efe00890df237142281879d56f31aa70" exitCode=0
Sep 11 12:03:48.010248 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.010217    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs"
Sep 11 12:03:48.010248 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.010236    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs" event={"ID":"e4138f6a-1f19-4f03-8177-d417cecfaba5","Type":"ContainerDied","Data":"054f579f72d5abb9ce40005d563cbf17efe00890df237142281879d56f31aa70"}
Sep 11 12:03:48.010492 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.010290    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs" event={"ID":"e4138f6a-1f19-4f03-8177-d417cecfaba5","Type":"ContainerDied","Data":"b1283383e8bce18cd8258274d4b65ae99f9bc3e734513700e8798f86ef539be2"}
Sep 11 12:03:48.010492 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.010319    2579 scope.go:117] "RemoveContainer" containerID="054f579f72d5abb9ce40005d563cbf17efe00890df237142281879d56f31aa70"
Sep 11 12:03:48.012632 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.012562    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4138f6a-1f19-4f03-8177-d417cecfaba5-workload-certs\") pod \"e4138f6a-1f19-4f03-8177-d417cecfaba5\" (UID: \"e4138f6a-1f19-4f03-8177-d417cecfaba5\") "
Sep 11 12:03:48.012809 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.012668    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4138f6a-1f19-4f03-8177-d417cecfaba5-istio-token\") pod \"e4138f6a-1f19-4f03-8177-d417cecfaba5\" (UID: \"e4138f6a-1f19-4f03-8177-d417cecfaba5\") "
Sep 11 12:03:48.012809 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.012717    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4138f6a-1f19-4f03-8177-d417cecfaba5-workload-socket\") pod \"e4138f6a-1f19-4f03-8177-d417cecfaba5\" (UID: \"e4138f6a-1f19-4f03-8177-d417cecfaba5\") "
Sep 11 12:03:48.012809 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.012753    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4138f6a-1f19-4f03-8177-d417cecfaba5-istio-data\") pod \"e4138f6a-1f19-4f03-8177-d417cecfaba5\" (UID: \"e4138f6a-1f19-4f03-8177-d417cecfaba5\") "
Sep 11 12:03:48.012809 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.012790    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bq8m6\" (UniqueName: \"kubernetes.io/projected/e4138f6a-1f19-4f03-8177-d417cecfaba5-kube-api-access-bq8m6\") pod \"e4138f6a-1f19-4f03-8177-d417cecfaba5\" (UID: \"e4138f6a-1f19-4f03-8177-d417cecfaba5\") "
Sep 11 12:03:48.013817 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.012839    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4138f6a-1f19-4f03-8177-d417cecfaba5-istio-envoy\") pod \"e4138f6a-1f19-4f03-8177-d417cecfaba5\" (UID: \"e4138f6a-1f19-4f03-8177-d417cecfaba5\") "
Sep 11 12:03:48.013817 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.012867    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4138f6a-1f19-4f03-8177-d417cecfaba5-istio-podinfo\") pod \"e4138f6a-1f19-4f03-8177-d417cecfaba5\" (UID: \"e4138f6a-1f19-4f03-8177-d417cecfaba5\") "
Sep 11 12:03:48.013817 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.012890    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4138f6a-1f19-4f03-8177-d417cecfaba5-istiod-ca-cert\") pod \"e4138f6a-1f19-4f03-8177-d417cecfaba5\" (UID: \"e4138f6a-1f19-4f03-8177-d417cecfaba5\") "
Sep 11 12:03:48.013817 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.012945    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4138f6a-1f19-4f03-8177-d417cecfaba5-credential-socket\") pod \"e4138f6a-1f19-4f03-8177-d417cecfaba5\" (UID: \"e4138f6a-1f19-4f03-8177-d417cecfaba5\") "
Sep 11 12:03:48.013817 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.013233    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4138f6a-1f19-4f03-8177-d417cecfaba5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e4138f6a-1f19-4f03-8177-d417cecfaba5" (UID: "e4138f6a-1f19-4f03-8177-d417cecfaba5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:03:48.014323 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.014295    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4138f6a-1f19-4f03-8177-d417cecfaba5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e4138f6a-1f19-4f03-8177-d417cecfaba5" (UID: "e4138f6a-1f19-4f03-8177-d417cecfaba5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:03:48.014497 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.014476    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4138f6a-1f19-4f03-8177-d417cecfaba5-istiod-ca-cert\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:03:48.014571 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.014502    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4138f6a-1f19-4f03-8177-d417cecfaba5-workload-certs\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:03:48.014571 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.014542    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4138f6a-1f19-4f03-8177-d417cecfaba5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e4138f6a-1f19-4f03-8177-d417cecfaba5" (UID: "e4138f6a-1f19-4f03-8177-d417cecfaba5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:03:48.014715 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.014663    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4138f6a-1f19-4f03-8177-d417cecfaba5-istio-data" (OuterVolumeSpecName: "istio-data") pod "e4138f6a-1f19-4f03-8177-d417cecfaba5" (UID: "e4138f6a-1f19-4f03-8177-d417cecfaba5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:03:48.015955 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.015912    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e4138f6a-1f19-4f03-8177-d417cecfaba5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e4138f6a-1f19-4f03-8177-d417cecfaba5" (UID: "e4138f6a-1f19-4f03-8177-d417cecfaba5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 12:03:48.016531 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.016447    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4138f6a-1f19-4f03-8177-d417cecfaba5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e4138f6a-1f19-4f03-8177-d417cecfaba5" (UID: "e4138f6a-1f19-4f03-8177-d417cecfaba5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:03:48.017901 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.017431    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4138f6a-1f19-4f03-8177-d417cecfaba5-istio-token" (OuterVolumeSpecName: "istio-token") pod "e4138f6a-1f19-4f03-8177-d417cecfaba5" (UID: "e4138f6a-1f19-4f03-8177-d417cecfaba5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:03:48.017901 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.017462    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4138f6a-1f19-4f03-8177-d417cecfaba5-kube-api-access-bq8m6" (OuterVolumeSpecName: "kube-api-access-bq8m6") pod "e4138f6a-1f19-4f03-8177-d417cecfaba5" (UID: "e4138f6a-1f19-4f03-8177-d417cecfaba5"). InnerVolumeSpecName "kube-api-access-bq8m6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:03:48.019046 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.019005    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4138f6a-1f19-4f03-8177-d417cecfaba5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e4138f6a-1f19-4f03-8177-d417cecfaba5" (UID: "e4138f6a-1f19-4f03-8177-d417cecfaba5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:03:48.040597 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.040459    2579 scope.go:117] "RemoveContainer" containerID="dc30ec3dfdac40c1b2e64f6737c020e8b40a68bbfe79c2d60ab38a41a63873bf"
Sep 11 12:03:48.050712 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.050691    2579 scope.go:117] "RemoveContainer" containerID="054f579f72d5abb9ce40005d563cbf17efe00890df237142281879d56f31aa70"
Sep 11 12:03:48.051043 ip-10-0-142-145 kubenswrapper[2579]: E0911 12:03:48.051020    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"054f579f72d5abb9ce40005d563cbf17efe00890df237142281879d56f31aa70\": container with ID starting with 054f579f72d5abb9ce40005d563cbf17efe00890df237142281879d56f31aa70 not found: ID does not exist" containerID="054f579f72d5abb9ce40005d563cbf17efe00890df237142281879d56f31aa70"
Sep 11 12:03:48.051132 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.051058    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"054f579f72d5abb9ce40005d563cbf17efe00890df237142281879d56f31aa70"} err="failed to get container status \"054f579f72d5abb9ce40005d563cbf17efe00890df237142281879d56f31aa70\": rpc error: code = NotFound desc = could not find container \"054f579f72d5abb9ce40005d563cbf17efe00890df237142281879d56f31aa70\": container with ID starting with 054f579f72d5abb9ce40005d563cbf17efe00890df237142281879d56f31aa70 not found: ID does not exist"
Sep 11 12:03:48.051132 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.051085    2579 scope.go:117] "RemoveContainer" containerID="dc30ec3dfdac40c1b2e64f6737c020e8b40a68bbfe79c2d60ab38a41a63873bf"
Sep 11 12:03:48.051444 ip-10-0-142-145 kubenswrapper[2579]: E0911 12:03:48.051422    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc30ec3dfdac40c1b2e64f6737c020e8b40a68bbfe79c2d60ab38a41a63873bf\": container with ID starting with dc30ec3dfdac40c1b2e64f6737c020e8b40a68bbfe79c2d60ab38a41a63873bf not found: ID does not exist" containerID="dc30ec3dfdac40c1b2e64f6737c020e8b40a68bbfe79c2d60ab38a41a63873bf"
Sep 11 12:03:48.051501 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.051453    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc30ec3dfdac40c1b2e64f6737c020e8b40a68bbfe79c2d60ab38a41a63873bf"} err="failed to get container status \"dc30ec3dfdac40c1b2e64f6737c020e8b40a68bbfe79c2d60ab38a41a63873bf\": rpc error: code = NotFound desc = could not find container \"dc30ec3dfdac40c1b2e64f6737c020e8b40a68bbfe79c2d60ab38a41a63873bf\": container with ID starting with dc30ec3dfdac40c1b2e64f6737c020e8b40a68bbfe79c2d60ab38a41a63873bf not found: ID does not exist"
Sep 11 12:03:48.115904 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.115867    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4138f6a-1f19-4f03-8177-d417cecfaba5-workload-socket\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:03:48.115904 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.115901    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4138f6a-1f19-4f03-8177-d417cecfaba5-istio-data\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:03:48.115904 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.115911    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bq8m6\" (UniqueName: \"kubernetes.io/projected/e4138f6a-1f19-4f03-8177-d417cecfaba5-kube-api-access-bq8m6\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:03:48.116178 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.115919    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4138f6a-1f19-4f03-8177-d417cecfaba5-istio-envoy\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:03:48.116178 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.115928    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4138f6a-1f19-4f03-8177-d417cecfaba5-istio-podinfo\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:03:48.116178 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.115937    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4138f6a-1f19-4f03-8177-d417cecfaba5-credential-socket\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:03:48.116178 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.115945    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4138f6a-1f19-4f03-8177-d417cecfaba5-istio-token\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:03:48.214730 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.214693    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:48.305441 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.305332    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-087015a2-6f994cd498-8lfmb"]
Sep 11 12:03:48.305812 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.305795    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4138f6a-1f19-4f03-8177-d417cecfaba5" containerName="istio-proxy"
Sep 11 12:03:48.305861 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.305814    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4138f6a-1f19-4f03-8177-d417cecfaba5" containerName="istio-proxy"
Sep 11 12:03:48.305861 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.305831    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4d53ce7-e017-436f-b2fe-848b4cdea538" containerName="payload-pre-processing"
Sep 11 12:03:48.305861 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.305837    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4d53ce7-e017-436f-b2fe-848b4cdea538" containerName="payload-pre-processing"
Sep 11 12:03:48.305861 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.305852    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e7c9c83-409d-4e70-8baf-dffef0609e88" containerName="payload-processing"
Sep 11 12:03:48.305861 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.305859    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e7c9c83-409d-4e70-8baf-dffef0609e88" containerName="payload-processing"
Sep 11 12:03:48.306000 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.305924    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4138f6a-1f19-4f03-8177-d417cecfaba5" containerName="istio-proxy"
Sep 11 12:03:48.306000 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.305932    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4d53ce7-e017-436f-b2fe-848b4cdea538" containerName="payload-pre-processing"
Sep 11 12:03:48.306000 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.305940    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e7c9c83-409d-4e70-8baf-dffef0609e88" containerName="payload-processing"
Sep 11 12:03:48.309424 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.309386    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-087015a2-6f994cd498-8lfmb"
Sep 11 12:03:48.313564 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.313540    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-087015a2\""
Sep 11 12:03:48.317408 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.317367    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8e71a0f-8d3c-480e-8249-755a8b072d83-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-087015a2-6f994cd498-8lfmb\" (UID: \"b8e71a0f-8d3c-480e-8249-755a8b072d83\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-087015a2-6f994cd498-8lfmb"
Sep 11 12:03:48.317531 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.317428    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbnbj\" (UniqueName: \"kubernetes.io/projected/b8e71a0f-8d3c-480e-8249-755a8b072d83-kube-api-access-hbnbj\") pod \"payload-pre-processing-e2e-ait-087015a2-6f994cd498-8lfmb\" (UID: \"b8e71a0f-8d3c-480e-8249-755a8b072d83\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-087015a2-6f994cd498-8lfmb"
Sep 11 12:03:48.324624 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.324600    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-087015a2-6f994cd498-8lfmb"]
Sep 11 12:03:48.329240 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.329212    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:48.368516 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.368484    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs"]
Sep 11 12:03:48.379900 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.379861    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-150d6af2-gw-openshift-default-6cbd88b48-wkkjs"]
Sep 11 12:03:48.418009 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.417975    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbnbj\" (UniqueName: \"kubernetes.io/projected/b8e71a0f-8d3c-480e-8249-755a8b072d83-kube-api-access-hbnbj\") pod \"payload-pre-processing-e2e-ait-087015a2-6f994cd498-8lfmb\" (UID: \"b8e71a0f-8d3c-480e-8249-755a8b072d83\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-087015a2-6f994cd498-8lfmb"
Sep 11 12:03:48.418164 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.418094    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8e71a0f-8d3c-480e-8249-755a8b072d83-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-087015a2-6f994cd498-8lfmb\" (UID: \"b8e71a0f-8d3c-480e-8249-755a8b072d83\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-087015a2-6f994cd498-8lfmb"
Sep 11 12:03:48.418820 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.418794    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8e71a0f-8d3c-480e-8249-755a8b072d83-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-087015a2-6f994cd498-8lfmb\" (UID: \"b8e71a0f-8d3c-480e-8249-755a8b072d83\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-087015a2-6f994cd498-8lfmb"
Sep 11 12:03:48.430028 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.430000    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbnbj\" (UniqueName: \"kubernetes.io/projected/b8e71a0f-8d3c-480e-8249-755a8b072d83-kube-api-access-hbnbj\") pod \"payload-pre-processing-e2e-ait-087015a2-6f994cd498-8lfmb\" (UID: \"b8e71a0f-8d3c-480e-8249-755a8b072d83\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-087015a2-6f994cd498-8lfmb"
Sep 11 12:03:48.476553 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.476520    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-087015a2-7456664d5f-mjjnb"]
Sep 11 12:03:48.477093 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.477077    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4138f6a-1f19-4f03-8177-d417cecfaba5" containerName="istio-proxy"
Sep 11 12:03:48.477144 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.477095    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4138f6a-1f19-4f03-8177-d417cecfaba5" containerName="istio-proxy"
Sep 11 12:03:48.477181 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.477158    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4138f6a-1f19-4f03-8177-d417cecfaba5" containerName="istio-proxy"
Sep 11 12:03:48.479514 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.479489    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-087015a2-7456664d5f-mjjnb"
Sep 11 12:03:48.483019 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.482996    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-087015a2-dockercfg-542tn\""
Sep 11 12:03:48.497208 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.497177    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-087015a2-7456664d5f-mjjnb"]
Sep 11 12:03:48.499505 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.499480    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:03:48.519269 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.519190    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqgps\" (UniqueName: \"kubernetes.io/projected/f0244682-c226-4b59-9d1c-8871b1bf2ba4-kube-api-access-rqgps\") pod \"payload-processing-e2e-ait-087015a2-7456664d5f-mjjnb\" (UID: \"f0244682-c226-4b59-9d1c-8871b1bf2ba4\") " pod="openshift-ingress/payload-processing-e2e-ait-087015a2-7456664d5f-mjjnb"
Sep 11 12:03:48.519269 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.519255    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0244682-c226-4b59-9d1c-8871b1bf2ba4-plugins-config-volume\") pod \"payload-processing-e2e-ait-087015a2-7456664d5f-mjjnb\" (UID: \"f0244682-c226-4b59-9d1c-8871b1bf2ba4\") " pod="openshift-ingress/payload-processing-e2e-ait-087015a2-7456664d5f-mjjnb"
Sep 11 12:03:48.577126 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.577037    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4138f6a-1f19-4f03-8177-d417cecfaba5" path="/var/lib/kubelet/pods/e4138f6a-1f19-4f03-8177-d417cecfaba5/volumes"
Sep 11 12:03:48.620564 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.620524    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-087015a2-6f994cd498-8lfmb"
Sep 11 12:03:48.620781 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.620759    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqgps\" (UniqueName: \"kubernetes.io/projected/f0244682-c226-4b59-9d1c-8871b1bf2ba4-kube-api-access-rqgps\") pod \"payload-processing-e2e-ait-087015a2-7456664d5f-mjjnb\" (UID: \"f0244682-c226-4b59-9d1c-8871b1bf2ba4\") " pod="openshift-ingress/payload-processing-e2e-ait-087015a2-7456664d5f-mjjnb"
Sep 11 12:03:48.620872 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.620812    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0244682-c226-4b59-9d1c-8871b1bf2ba4-plugins-config-volume\") pod \"payload-processing-e2e-ait-087015a2-7456664d5f-mjjnb\" (UID: \"f0244682-c226-4b59-9d1c-8871b1bf2ba4\") " pod="openshift-ingress/payload-processing-e2e-ait-087015a2-7456664d5f-mjjnb"
Sep 11 12:03:48.621501 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.621478    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0244682-c226-4b59-9d1c-8871b1bf2ba4-plugins-config-volume\") pod \"payload-processing-e2e-ait-087015a2-7456664d5f-mjjnb\" (UID: \"f0244682-c226-4b59-9d1c-8871b1bf2ba4\") " pod="openshift-ingress/payload-processing-e2e-ait-087015a2-7456664d5f-mjjnb"
Sep 11 12:03:48.637993 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.637973    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqgps\" (UniqueName: \"kubernetes.io/projected/f0244682-c226-4b59-9d1c-8871b1bf2ba4-kube-api-access-rqgps\") pod \"payload-processing-e2e-ait-087015a2-7456664d5f-mjjnb\" (UID: \"f0244682-c226-4b59-9d1c-8871b1bf2ba4\") " pod="openshift-ingress/payload-processing-e2e-ait-087015a2-7456664d5f-mjjnb"
Sep 11 12:03:48.790586 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.790546    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-087015a2-7456664d5f-mjjnb"
Sep 11 12:03:48.896799 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:48.896762    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-087015a2-6f994cd498-8lfmb"]
Sep 11 12:03:48.897759 ip-10-0-142-145 kubenswrapper[2579]: W0911 12:03:48.897734    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8e71a0f_8d3c_480e_8249_755a8b072d83.slice/crio-7fddde39f9fa8d3f2563c1a76a7e01b7e80ec20b529eb20c95989de276edf228 WatchSource:0}: Error finding container 7fddde39f9fa8d3f2563c1a76a7e01b7e80ec20b529eb20c95989de276edf228: Status 404 returned error can't find the container with id 7fddde39f9fa8d3f2563c1a76a7e01b7e80ec20b529eb20c95989de276edf228
Sep 11 12:03:49.024893 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:49.024851    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-087015a2-6f994cd498-8lfmb" event={"ID":"b8e71a0f-8d3c-480e-8249-755a8b072d83","Type":"ContainerStarted","Data":"bf61a15ba60ce8de696cd1b45f16b0b9f344b11d65b8dd95d228fde666855204"}
Sep 11 12:03:49.025064 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:49.024947    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-087015a2-6f994cd498-8lfmb"
Sep 11 12:03:49.025064 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:49.024967    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-087015a2-6f994cd498-8lfmb" event={"ID":"b8e71a0f-8d3c-480e-8249-755a8b072d83","Type":"ContainerStarted","Data":"7fddde39f9fa8d3f2563c1a76a7e01b7e80ec20b529eb20c95989de276edf228"}
Sep 11 12:03:49.071065 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:49.071009    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-087015a2-6f994cd498-8lfmb" podStartSLOduration=1.070991194 podStartE2EDuration="1.070991194s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:03:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:03:49.06993078 +0000 UTC m=+932.967372524" watchObservedRunningTime="2026-09-11 12:03:49.070991194 +0000 UTC m=+932.968432939"
Sep 11 12:03:49.084149 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:49.084113    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-087015a2-7456664d5f-mjjnb"]
Sep 11 12:03:49.087678 ip-10-0-142-145 kubenswrapper[2579]: W0911 12:03:49.087647    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0244682_c226_4b59_9d1c_8871b1bf2ba4.slice/crio-8376f6c2b7b90a8a6d806cf12b74a8f62d4a08b5e49eff1be9421f1446c5f4f6 WatchSource:0}: Error finding container 8376f6c2b7b90a8a6d806cf12b74a8f62d4a08b5e49eff1be9421f1446c5f4f6: Status 404 returned error can't find the container with id 8376f6c2b7b90a8a6d806cf12b74a8f62d4a08b5e49eff1be9421f1446c5f4f6
Sep 11 12:03:50.032749 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:50.032713    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-087015a2-7456664d5f-mjjnb" event={"ID":"f0244682-c226-4b59-9d1c-8871b1bf2ba4","Type":"ContainerStarted","Data":"3d7e42c7ea1f0603aafe9e3b75f96910453a60c047afe68f10d5b7241d7443a3"}
Sep 11 12:03:50.033183 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:50.032778    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-087015a2-7456664d5f-mjjnb" event={"ID":"f0244682-c226-4b59-9d1c-8871b1bf2ba4","Type":"ContainerStarted","Data":"8376f6c2b7b90a8a6d806cf12b74a8f62d4a08b5e49eff1be9421f1446c5f4f6"}
Sep 11 12:03:50.033183 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:50.032960    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-087015a2-7456664d5f-mjjnb"
Sep 11 12:03:50.063210 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:50.063159    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-087015a2-7456664d5f-mjjnb" podStartSLOduration=2.063143908 podStartE2EDuration="2.063143908s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:03:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:03:50.060899707 +0000 UTC m=+933.958341452" watchObservedRunningTime="2026-09-11 12:03:50.063143908 +0000 UTC m=+933.960585652"
Sep 11 12:03:51.334679 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:03:51.334647    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:00.035518 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:00.035488    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-087015a2-6f994cd498-8lfmb"
Sep 11 12:04:01.041095 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:01.041060    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-087015a2-7456664d5f-mjjnb"
Sep 11 12:04:04.215760 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:04.215725    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:05.974495 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:05.974450    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:04:09.525693 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:09.525658    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:10.350458 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:10.350422    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:11.513764 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:11.513733    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:12.622812 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:12.622779    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:18.558679 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:18.558635    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:18.715548 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:18.715517    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-087015a2-7456664d5f-mjjnb"]
Sep 11 12:04:18.715752 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:18.715731    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-087015a2-7456664d5f-mjjnb" podUID="f0244682-c226-4b59-9d1c-8871b1bf2ba4" containerName="payload-processing" containerID="cri-o://3d7e42c7ea1f0603aafe9e3b75f96910453a60c047afe68f10d5b7241d7443a3" gracePeriod=30
Sep 11 12:04:18.739201 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:18.739172    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-087015a2-6f994cd498-8lfmb"]
Sep 11 12:04:18.739428 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:18.739365    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-087015a2-6f994cd498-8lfmb" podUID="b8e71a0f-8d3c-480e-8249-755a8b072d83" containerName="payload-pre-processing" containerID="cri-o://bf61a15ba60ce8de696cd1b45f16b0b9f344b11d65b8dd95d228fde666855204" gracePeriod=30
Sep 11 12:04:18.837275 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:18.837190    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:19.003690 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:19.003662    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:19.027860 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:19.027832    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-087015a2-7456664d5f-mjjnb"
Sep 11 12:04:19.031123 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:19.031101    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-087015a2-6f994cd498-8lfmb"
Sep 11 12:04:19.124917 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:19.124880    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8e71a0f-8d3c-480e-8249-755a8b072d83-plugins-config-volume\") pod \"b8e71a0f-8d3c-480e-8249-755a8b072d83\" (UID: \"b8e71a0f-8d3c-480e-8249-755a8b072d83\") "
Sep 11 12:04:19.125258 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:19.125235    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hbnbj\" (UniqueName: \"kubernetes.io/projected/b8e71a0f-8d3c-480e-8249-755a8b072d83-kube-api-access-hbnbj\") pod \"b8e71a0f-8d3c-480e-8249-755a8b072d83\" (UID: \"b8e71a0f-8d3c-480e-8249-755a8b072d83\") "
Sep 11 12:04:19.125627 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:19.125384    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rqgps\" (UniqueName: \"kubernetes.io/projected/f0244682-c226-4b59-9d1c-8871b1bf2ba4-kube-api-access-rqgps\") pod \"f0244682-c226-4b59-9d1c-8871b1bf2ba4\" (UID: \"f0244682-c226-4b59-9d1c-8871b1bf2ba4\") "
Sep 11 12:04:19.125791 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:19.125772    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0244682-c226-4b59-9d1c-8871b1bf2ba4-plugins-config-volume\") pod \"f0244682-c226-4b59-9d1c-8871b1bf2ba4\" (UID: \"f0244682-c226-4b59-9d1c-8871b1bf2ba4\") "
Sep 11 12:04:19.126603 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:19.126576    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0244682-c226-4b59-9d1c-8871b1bf2ba4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f0244682-c226-4b59-9d1c-8871b1bf2ba4" (UID: "f0244682-c226-4b59-9d1c-8871b1bf2ba4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:04:19.126752 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:19.126643    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8e71a0f-8d3c-480e-8249-755a8b072d83-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b8e71a0f-8d3c-480e-8249-755a8b072d83" (UID: "b8e71a0f-8d3c-480e-8249-755a8b072d83"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:04:19.130475 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:19.130447    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8e71a0f-8d3c-480e-8249-755a8b072d83-kube-api-access-hbnbj" (OuterVolumeSpecName: "kube-api-access-hbnbj") pod "b8e71a0f-8d3c-480e-8249-755a8b072d83" (UID: "b8e71a0f-8d3c-480e-8249-755a8b072d83"). InnerVolumeSpecName "kube-api-access-hbnbj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:04:19.130938 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:19.130480    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0244682-c226-4b59-9d1c-8871b1bf2ba4-kube-api-access-rqgps" (OuterVolumeSpecName: "kube-api-access-rqgps") pod "f0244682-c226-4b59-9d1c-8871b1bf2ba4" (UID: "f0244682-c226-4b59-9d1c-8871b1bf2ba4"). InnerVolumeSpecName "kube-api-access-rqgps". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:04:19.182251 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:19.182206    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:19.197613 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:19.197578    2579 generic.go:356] "Generic (PLEG): container finished" podID="b8e71a0f-8d3c-480e-8249-755a8b072d83" containerID="bf61a15ba60ce8de696cd1b45f16b0b9f344b11d65b8dd95d228fde666855204" exitCode=0
Sep 11 12:04:19.197803 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:19.197637    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-087015a2-6f994cd498-8lfmb"
Sep 11 12:04:19.197803 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:19.197650    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-087015a2-6f994cd498-8lfmb" event={"ID":"b8e71a0f-8d3c-480e-8249-755a8b072d83","Type":"ContainerDied","Data":"bf61a15ba60ce8de696cd1b45f16b0b9f344b11d65b8dd95d228fde666855204"}
Sep 11 12:04:19.197803 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:19.197734    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-087015a2-6f994cd498-8lfmb" event={"ID":"b8e71a0f-8d3c-480e-8249-755a8b072d83","Type":"ContainerDied","Data":"7fddde39f9fa8d3f2563c1a76a7e01b7e80ec20b529eb20c95989de276edf228"}
Sep 11 12:04:19.197803 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:19.197762    2579 scope.go:117] "RemoveContainer" containerID="bf61a15ba60ce8de696cd1b45f16b0b9f344b11d65b8dd95d228fde666855204"
Sep 11 12:04:19.199524 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:19.199499    2579 generic.go:356] "Generic (PLEG): container finished" podID="f0244682-c226-4b59-9d1c-8871b1bf2ba4" containerID="3d7e42c7ea1f0603aafe9e3b75f96910453a60c047afe68f10d5b7241d7443a3" exitCode=0
Sep 11 12:04:19.199652 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:19.199561    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-087015a2-7456664d5f-mjjnb"
Sep 11 12:04:19.199652 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:19.199581    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-087015a2-7456664d5f-mjjnb" event={"ID":"f0244682-c226-4b59-9d1c-8871b1bf2ba4","Type":"ContainerDied","Data":"3d7e42c7ea1f0603aafe9e3b75f96910453a60c047afe68f10d5b7241d7443a3"}
Sep 11 12:04:19.199741 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:19.199650    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-087015a2-7456664d5f-mjjnb" event={"ID":"f0244682-c226-4b59-9d1c-8871b1bf2ba4","Type":"ContainerDied","Data":"8376f6c2b7b90a8a6d806cf12b74a8f62d4a08b5e49eff1be9421f1446c5f4f6"}
Sep 11 12:04:19.207445 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:19.207427    2579 scope.go:117] "RemoveContainer" containerID="bf61a15ba60ce8de696cd1b45f16b0b9f344b11d65b8dd95d228fde666855204"
Sep 11 12:04:19.207731 ip-10-0-142-145 kubenswrapper[2579]: E0911 12:04:19.207715    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf61a15ba60ce8de696cd1b45f16b0b9f344b11d65b8dd95d228fde666855204\": container with ID starting with bf61a15ba60ce8de696cd1b45f16b0b9f344b11d65b8dd95d228fde666855204 not found: ID does not exist" containerID="bf61a15ba60ce8de696cd1b45f16b0b9f344b11d65b8dd95d228fde666855204"
Sep 11 12:04:19.207787 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:19.207740    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf61a15ba60ce8de696cd1b45f16b0b9f344b11d65b8dd95d228fde666855204"} err="failed to get container status \"bf61a15ba60ce8de696cd1b45f16b0b9f344b11d65b8dd95d228fde666855204\": rpc error: code = NotFound desc = could not find container \"bf61a15ba60ce8de696cd1b45f16b0b9f344b11d65b8dd95d228fde666855204\": container with ID starting with bf61a15ba60ce8de696cd1b45f16b0b9f344b11d65b8dd95d228fde666855204 not found: ID does not exist"
Sep 11 12:04:19.207787 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:19.207757    2579 scope.go:117] "RemoveContainer" containerID="3d7e42c7ea1f0603aafe9e3b75f96910453a60c047afe68f10d5b7241d7443a3"
Sep 11 12:04:19.215826 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:19.215809    2579 scope.go:117] "RemoveContainer" containerID="3d7e42c7ea1f0603aafe9e3b75f96910453a60c047afe68f10d5b7241d7443a3"
Sep 11 12:04:19.216089 ip-10-0-142-145 kubenswrapper[2579]: E0911 12:04:19.216069    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d7e42c7ea1f0603aafe9e3b75f96910453a60c047afe68f10d5b7241d7443a3\": container with ID starting with 3d7e42c7ea1f0603aafe9e3b75f96910453a60c047afe68f10d5b7241d7443a3 not found: ID does not exist" containerID="3d7e42c7ea1f0603aafe9e3b75f96910453a60c047afe68f10d5b7241d7443a3"
Sep 11 12:04:19.216137 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:19.216095    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d7e42c7ea1f0603aafe9e3b75f96910453a60c047afe68f10d5b7241d7443a3"} err="failed to get container status \"3d7e42c7ea1f0603aafe9e3b75f96910453a60c047afe68f10d5b7241d7443a3\": rpc error: code = NotFound desc = could not find container \"3d7e42c7ea1f0603aafe9e3b75f96910453a60c047afe68f10d5b7241d7443a3\": container with ID starting with 3d7e42c7ea1f0603aafe9e3b75f96910453a60c047afe68f10d5b7241d7443a3 not found: ID does not exist"
Sep 11 12:04:19.226953 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:19.226922    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hbnbj\" (UniqueName: \"kubernetes.io/projected/b8e71a0f-8d3c-480e-8249-755a8b072d83-kube-api-access-hbnbj\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:04:19.226953 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:19.226950    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rqgps\" (UniqueName: \"kubernetes.io/projected/f0244682-c226-4b59-9d1c-8871b1bf2ba4-kube-api-access-rqgps\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:04:19.226953 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:19.226961    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0244682-c226-4b59-9d1c-8871b1bf2ba4-plugins-config-volume\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:04:19.227166 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:19.226969    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8e71a0f-8d3c-480e-8249-755a8b072d83-plugins-config-volume\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:04:19.251859 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:19.251828    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-087015a2-6f994cd498-8lfmb"]
Sep 11 12:04:19.256227 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:19.256199    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-087015a2-6f994cd498-8lfmb"]
Sep 11 12:04:19.282150 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:19.282115    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-087015a2-7456664d5f-mjjnb"]
Sep 11 12:04:19.285007 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:19.284975    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-087015a2-7456664d5f-mjjnb"]
Sep 11 12:04:19.342895 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:19.342860    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:19.668146 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:19.668110    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:20.506153 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:20.506124    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:20.576563 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:20.576524    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8e71a0f-8d3c-480e-8249-755a8b072d83" path="/var/lib/kubelet/pods/b8e71a0f-8d3c-480e-8249-755a8b072d83/volumes"
Sep 11 12:04:20.577053 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:20.577031    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0244682-c226-4b59-9d1c-8871b1bf2ba4" path="/var/lib/kubelet/pods/f0244682-c226-4b59-9d1c-8871b1bf2ba4/volumes"
Sep 11 12:04:20.613824 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:20.613789    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:21.025859 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:21.025826    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-cdec04-585c76b54c-544tc"]
Sep 11 12:04:21.026309 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:21.026039    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-cdec04-585c76b54c-544tc" podUID="429c86e4-a876-4029-9eaa-264a892cef11" containerName="maas-api" containerID="cri-o://ff365fba54a79fcb6b33b50a13f757af2b20522373f5a164d3d9c91c911dd5b4" gracePeriod=30
Sep 11 12:04:21.067241 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:21.067207    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:21.215454 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:21.215423    2579 generic.go:356] "Generic (PLEG): container finished" podID="429c86e4-a876-4029-9eaa-264a892cef11" containerID="ff365fba54a79fcb6b33b50a13f757af2b20522373f5a164d3d9c91c911dd5b4" exitCode=0
Sep 11 12:04:21.215604 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:21.215498    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-cdec04-585c76b54c-544tc" event={"ID":"429c86e4-a876-4029-9eaa-264a892cef11","Type":"ContainerDied","Data":"ff365fba54a79fcb6b33b50a13f757af2b20522373f5a164d3d9c91c911dd5b4"}
Sep 11 12:04:21.304521 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:21.304445    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-cdec04-585c76b54c-544tc"
Sep 11 12:04:21.305802 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:21.305769    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:21.449679 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:21.449649    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/429c86e4-a876-4029-9eaa-264a892cef11-metrics-certs\") pod \"429c86e4-a876-4029-9eaa-264a892cef11\" (UID: \"429c86e4-a876-4029-9eaa-264a892cef11\") "
Sep 11 12:04:21.449855 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:21.449701    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/429c86e4-a876-4029-9eaa-264a892cef11-maas-api-tls\") pod \"429c86e4-a876-4029-9eaa-264a892cef11\" (UID: \"429c86e4-a876-4029-9eaa-264a892cef11\") "
Sep 11 12:04:21.449855 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:21.449746    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/429c86e4-a876-4029-9eaa-264a892cef11-openshift-service-ca\") pod \"429c86e4-a876-4029-9eaa-264a892cef11\" (UID: \"429c86e4-a876-4029-9eaa-264a892cef11\") "
Sep 11 12:04:21.449855 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:21.449809    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zxswk\" (UniqueName: \"kubernetes.io/projected/429c86e4-a876-4029-9eaa-264a892cef11-kube-api-access-zxswk\") pod \"429c86e4-a876-4029-9eaa-264a892cef11\" (UID: \"429c86e4-a876-4029-9eaa-264a892cef11\") "
Sep 11 12:04:21.450139 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:21.450114    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/429c86e4-a876-4029-9eaa-264a892cef11-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "429c86e4-a876-4029-9eaa-264a892cef11" (UID: "429c86e4-a876-4029-9eaa-264a892cef11"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:04:21.451990 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:21.451965    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/429c86e4-a876-4029-9eaa-264a892cef11-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "429c86e4-a876-4029-9eaa-264a892cef11" (UID: "429c86e4-a876-4029-9eaa-264a892cef11"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:04:21.452123 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:21.452036    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/429c86e4-a876-4029-9eaa-264a892cef11-kube-api-access-zxswk" (OuterVolumeSpecName: "kube-api-access-zxswk") pod "429c86e4-a876-4029-9eaa-264a892cef11" (UID: "429c86e4-a876-4029-9eaa-264a892cef11"). InnerVolumeSpecName "kube-api-access-zxswk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:04:21.452123 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:21.452078    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/429c86e4-a876-4029-9eaa-264a892cef11-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "429c86e4-a876-4029-9eaa-264a892cef11" (UID: "429c86e4-a876-4029-9eaa-264a892cef11"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:04:21.474495 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:21.474455    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268"]
Sep 11 12:04:21.474644 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:21.474458    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:21.474917 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:21.474903    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8e71a0f-8d3c-480e-8249-755a8b072d83" containerName="payload-pre-processing"
Sep 11 12:04:21.474966 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:21.474920    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8e71a0f-8d3c-480e-8249-755a8b072d83" containerName="payload-pre-processing"
Sep 11 12:04:21.474966 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:21.474935    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="429c86e4-a876-4029-9eaa-264a892cef11" containerName="maas-api"
Sep 11 12:04:21.474966 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:21.474942    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="429c86e4-a876-4029-9eaa-264a892cef11" containerName="maas-api"
Sep 11 12:04:21.474966 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:21.474966    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0244682-c226-4b59-9d1c-8871b1bf2ba4" containerName="payload-processing"
Sep 11 12:04:21.475095 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:21.474972    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0244682-c226-4b59-9d1c-8871b1bf2ba4" containerName="payload-processing"
Sep 11 12:04:21.475095 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:21.475024    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0244682-c226-4b59-9d1c-8871b1bf2ba4" containerName="payload-processing"
Sep 11 12:04:21.475095 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:21.475032    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="429c86e4-a876-4029-9eaa-264a892cef11" containerName="maas-api"
Sep 11 12:04:21.475095 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:21.475040    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8e71a0f-8d3c-480e-8249-755a8b072d83" containerName="payload-pre-processing"
Sep 11 12:04:21.477367 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:21.477351    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268"
Sep 11 12:04:21.482079 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:21.482061    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-0eb76ba5-openshift-default-dockercfg-bjnj7\""
Sep 11 12:04:21.506283 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:21.506249    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268"]
Sep 11 12:04:21.557363 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:21.557272    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zxswk\" (UniqueName: \"kubernetes.io/projected/429c86e4-a876-4029-9eaa-264a892cef11-kube-api-access-zxswk\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:04:21.557363 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:21.557307    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/429c86e4-a876-4029-9eaa-264a892cef11-metrics-certs\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:04:21.557363 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:21.557321    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/429c86e4-a876-4029-9eaa-264a892cef11-maas-api-tls\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:04:21.557363 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:21.557331    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/429c86e4-a876-4029-9eaa-264a892cef11-openshift-service-ca\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:04:21.626547 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:21.626507    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:21.658474 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:21.658435    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fca7bbc3-b185-48d3-bc1d-91a127f33876-workload-certs\") pod \"e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268\" (UID: \"fca7bbc3-b185-48d3-bc1d-91a127f33876\") " pod="openshift-ingress/e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268"
Sep 11 12:04:21.658474 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:21.658474    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6g997\" (UniqueName: \"kubernetes.io/projected/fca7bbc3-b185-48d3-bc1d-91a127f33876-kube-api-access-6g997\") pod \"e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268\" (UID: \"fca7bbc3-b185-48d3-bc1d-91a127f33876\") " pod="openshift-ingress/e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268"
Sep 11 12:04:21.658707 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:21.658509    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fca7bbc3-b185-48d3-bc1d-91a127f33876-workload-socket\") pod \"e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268\" (UID: \"fca7bbc3-b185-48d3-bc1d-91a127f33876\") " pod="openshift-ingress/e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268"
Sep 11 12:04:21.658707 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:21.658527    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fca7bbc3-b185-48d3-bc1d-91a127f33876-istiod-ca-cert\") pod \"e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268\" (UID: \"fca7bbc3-b185-48d3-bc1d-91a127f33876\") " pod="openshift-ingress/e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268"
Sep 11 12:04:21.658707 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:21.658558    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fca7bbc3-b185-48d3-bc1d-91a127f33876-istio-envoy\") pod \"e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268\" (UID: \"fca7bbc3-b185-48d3-bc1d-91a127f33876\") " pod="openshift-ingress/e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268"
Sep 11 12:04:21.658707 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:21.658622    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fca7bbc3-b185-48d3-bc1d-91a127f33876-istio-podinfo\") pod \"e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268\" (UID: \"fca7bbc3-b185-48d3-bc1d-91a127f33876\") " pod="openshift-ingress/e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268"
Sep 11 12:04:21.658707 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:21.658680    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fca7bbc3-b185-48d3-bc1d-91a127f33876-credential-socket\") pod \"e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268\" (UID: \"fca7bbc3-b185-48d3-bc1d-91a127f33876\") " pod="openshift-ingress/e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268"
Sep 11 12:04:21.658883 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:21.658748    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fca7bbc3-b185-48d3-bc1d-91a127f33876-istio-data\") pod \"e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268\" (UID: \"fca7bbc3-b185-48d3-bc1d-91a127f33876\") " pod="openshift-ingress/e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268"
Sep 11 12:04:21.658883 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:21.658772    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fca7bbc3-b185-48d3-bc1d-91a127f33876-istio-token\") pod \"e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268\" (UID: \"fca7bbc3-b185-48d3-bc1d-91a127f33876\") " pod="openshift-ingress/e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268"
Sep 11 12:04:21.730570 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:21.730530    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:21.759850 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:21.759808    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fca7bbc3-b185-48d3-bc1d-91a127f33876-credential-socket\") pod \"e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268\" (UID: \"fca7bbc3-b185-48d3-bc1d-91a127f33876\") " pod="openshift-ingress/e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268"
Sep 11 12:04:21.760705 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:21.759897    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fca7bbc3-b185-48d3-bc1d-91a127f33876-istio-data\") pod \"e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268\" (UID: \"fca7bbc3-b185-48d3-bc1d-91a127f33876\") " pod="openshift-ingress/e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268"
Sep 11 12:04:21.760705 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:21.759929    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fca7bbc3-b185-48d3-bc1d-91a127f33876-istio-token\") pod \"e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268\" (UID: \"fca7bbc3-b185-48d3-bc1d-91a127f33876\") " pod="openshift-ingress/e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268"
Sep 11 12:04:21.760705 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:21.760008    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fca7bbc3-b185-48d3-bc1d-91a127f33876-workload-certs\") pod \"e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268\" (UID: \"fca7bbc3-b185-48d3-bc1d-91a127f33876\") " pod="openshift-ingress/e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268"
Sep 11 12:04:21.760705 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:21.760033    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6g997\" (UniqueName: \"kubernetes.io/projected/fca7bbc3-b185-48d3-bc1d-91a127f33876-kube-api-access-6g997\") pod \"e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268\" (UID: \"fca7bbc3-b185-48d3-bc1d-91a127f33876\") " pod="openshift-ingress/e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268"
Sep 11 12:04:21.760705 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:21.760086    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fca7bbc3-b185-48d3-bc1d-91a127f33876-workload-socket\") pod \"e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268\" (UID: \"fca7bbc3-b185-48d3-bc1d-91a127f33876\") " pod="openshift-ingress/e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268"
Sep 11 12:04:21.760705 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:21.760115    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fca7bbc3-b185-48d3-bc1d-91a127f33876-istiod-ca-cert\") pod \"e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268\" (UID: \"fca7bbc3-b185-48d3-bc1d-91a127f33876\") " pod="openshift-ingress/e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268"
Sep 11 12:04:21.760705 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:21.760162    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fca7bbc3-b185-48d3-bc1d-91a127f33876-istio-envoy\") pod \"e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268\" (UID: \"fca7bbc3-b185-48d3-bc1d-91a127f33876\") " pod="openshift-ingress/e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268"
Sep 11 12:04:21.760705 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:21.760195    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fca7bbc3-b185-48d3-bc1d-91a127f33876-istio-podinfo\") pod \"e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268\" (UID: \"fca7bbc3-b185-48d3-bc1d-91a127f33876\") " pod="openshift-ingress/e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268"
Sep 11 12:04:21.760705 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:21.760369    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fca7bbc3-b185-48d3-bc1d-91a127f33876-credential-socket\") pod \"e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268\" (UID: \"fca7bbc3-b185-48d3-bc1d-91a127f33876\") " pod="openshift-ingress/e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268"
Sep 11 12:04:21.760705 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:21.760423    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fca7bbc3-b185-48d3-bc1d-91a127f33876-istio-data\") pod \"e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268\" (UID: \"fca7bbc3-b185-48d3-bc1d-91a127f33876\") " pod="openshift-ingress/e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268"
Sep 11 12:04:21.760705 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:21.760465    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fca7bbc3-b185-48d3-bc1d-91a127f33876-workload-certs\") pod \"e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268\" (UID: \"fca7bbc3-b185-48d3-bc1d-91a127f33876\") " pod="openshift-ingress/e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268"
Sep 11 12:04:21.760705 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:21.760690    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fca7bbc3-b185-48d3-bc1d-91a127f33876-workload-socket\") pod \"e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268\" (UID: \"fca7bbc3-b185-48d3-bc1d-91a127f33876\") " pod="openshift-ingress/e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268"
Sep 11 12:04:21.761202 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:21.760898    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fca7bbc3-b185-48d3-bc1d-91a127f33876-istiod-ca-cert\") pod \"e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268\" (UID: \"fca7bbc3-b185-48d3-bc1d-91a127f33876\") " pod="openshift-ingress/e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268"
Sep 11 12:04:21.763274 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:21.763244    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fca7bbc3-b185-48d3-bc1d-91a127f33876-istio-envoy\") pod \"e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268\" (UID: \"fca7bbc3-b185-48d3-bc1d-91a127f33876\") " pod="openshift-ingress/e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268"
Sep 11 12:04:21.763425 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:21.763407    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fca7bbc3-b185-48d3-bc1d-91a127f33876-istio-podinfo\") pod \"e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268\" (UID: \"fca7bbc3-b185-48d3-bc1d-91a127f33876\") " pod="openshift-ingress/e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268"
Sep 11 12:04:21.770365 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:21.770334    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fca7bbc3-b185-48d3-bc1d-91a127f33876-istio-token\") pod \"e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268\" (UID: \"fca7bbc3-b185-48d3-bc1d-91a127f33876\") " pod="openshift-ingress/e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268"
Sep 11 12:04:21.770483 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:21.770468    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6g997\" (UniqueName: \"kubernetes.io/projected/fca7bbc3-b185-48d3-bc1d-91a127f33876-kube-api-access-6g997\") pod \"e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268\" (UID: \"fca7bbc3-b185-48d3-bc1d-91a127f33876\") " pod="openshift-ingress/e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268"
Sep 11 12:04:21.788345 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:21.788317    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268"
Sep 11 12:04:22.055357 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:22.055323    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:22.068232 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:22.068200    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268"]
Sep 11 12:04:22.069703 ip-10-0-142-145 kubenswrapper[2579]: W0911 12:04:22.069677    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfca7bbc3_b185_48d3_bc1d_91a127f33876.slice/crio-f4f8069f0bf7aa312b8e6f4889d3c215281b9fc7813e09c91e181e794d8479af WatchSource:0}: Error finding container f4f8069f0bf7aa312b8e6f4889d3c215281b9fc7813e09c91e181e794d8479af: Status 404 returned error can't find the container with id f4f8069f0bf7aa312b8e6f4889d3c215281b9fc7813e09c91e181e794d8479af
Sep 11 12:04:22.072142 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:22.072103    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 12:04:22.072213 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:22.072199    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 12:04:22.072264 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:22.072247    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 12:04:22.223462 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:22.223426    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268" event={"ID":"fca7bbc3-b185-48d3-bc1d-91a127f33876","Type":"ContainerStarted","Data":"67ca1480363e7556fee8bd4797abb6da52c240b80602ef30aefa2e4b4325146c"}
Sep 11 12:04:22.223669 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:22.223488    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268" event={"ID":"fca7bbc3-b185-48d3-bc1d-91a127f33876","Type":"ContainerStarted","Data":"f4f8069f0bf7aa312b8e6f4889d3c215281b9fc7813e09c91e181e794d8479af"}
Sep 11 12:04:22.225354 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:22.225322    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-cdec04-585c76b54c-544tc" event={"ID":"429c86e4-a876-4029-9eaa-264a892cef11","Type":"ContainerDied","Data":"2f9f75c0cc084e525fd1536fe8252a17ab3389e7533353865638afa12cef72ad"}
Sep 11 12:04:22.225354 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:22.225344    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-cdec04-585c76b54c-544tc"
Sep 11 12:04:22.225581 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:22.225421    2579 scope.go:117] "RemoveContainer" containerID="ff365fba54a79fcb6b33b50a13f757af2b20522373f5a164d3d9c91c911dd5b4"
Sep 11 12:04:22.253690 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:22.253605    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268" podStartSLOduration=1.253582096 podStartE2EDuration="1.253582096s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:04:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:04:22.25171722 +0000 UTC m=+966.149158966" watchObservedRunningTime="2026-09-11 12:04:22.253582096 +0000 UTC m=+966.151023844"
Sep 11 12:04:22.269214 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:22.269175    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-cdec04-585c76b54c-544tc"]
Sep 11 12:04:22.279298 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:22.279266    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-cdec04-585c76b54c-544tc"]
Sep 11 12:04:22.585602 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:22.585567    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="429c86e4-a876-4029-9eaa-264a892cef11" path="/var/lib/kubelet/pods/429c86e4-a876-4029-9eaa-264a892cef11/volumes"
Sep 11 12:04:22.789162 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:22.789121    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268"
Sep 11 12:04:22.818489 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:22.818449    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:23.650355 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:23.650277    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:23.761817 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:23.761780    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:23.794015 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:23.793982    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268"
Sep 11 12:04:24.237731 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:24.237698    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268"
Sep 11 12:04:24.238560 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:24.238539    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268"
Sep 11 12:04:24.905578 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:24.905545    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:25.014519 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:25.014485    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:25.142987 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:25.142951    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:25.565630 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:25.565601    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:25.662716 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:25.662681    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:25.671571 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:25.671541    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-0eb76ba5-785cb7b9b6-c4g2f"]
Sep 11 12:04:25.674924 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:25.674905    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-0eb76ba5-785cb7b9b6-c4g2f"
Sep 11 12:04:25.678576 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:25.678548    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-0eb76ba5\""
Sep 11 12:04:25.689141 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:25.689119    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-0eb76ba5-785cb7b9b6-c4g2f"]
Sep 11 12:04:25.694123 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:25.694100    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-0eb76ba5-d9d447497-hc85x"]
Sep 11 12:04:25.696668 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:25.696651    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-0eb76ba5-d9d447497-hc85x"
Sep 11 12:04:25.700138 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:25.700120    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-0eb76ba5-dockercfg-2t9jn\""
Sep 11 12:04:25.707922 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:25.707899    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d8f10d4-ffec-4eca-9037-4958de3cfe2c-plugins-config-volume\") pod \"payload-processing-e2e-derive-0eb76ba5-d9d447497-hc85x\" (UID: \"9d8f10d4-ffec-4eca-9037-4958de3cfe2c\") " pod="openshift-ingress/payload-processing-e2e-derive-0eb76ba5-d9d447497-hc85x"
Sep 11 12:04:25.707998 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:25.707954    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkcrf\" (UniqueName: \"kubernetes.io/projected/9d8f10d4-ffec-4eca-9037-4958de3cfe2c-kube-api-access-pkcrf\") pod \"payload-processing-e2e-derive-0eb76ba5-d9d447497-hc85x\" (UID: \"9d8f10d4-ffec-4eca-9037-4958de3cfe2c\") " pod="openshift-ingress/payload-processing-e2e-derive-0eb76ba5-d9d447497-hc85x"
Sep 11 12:04:25.708051 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:25.708003    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d6ba210-7964-459d-a6d2-0732097b0fa0-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-0eb76ba5-785cb7b9b6-c4g2f\" (UID: \"2d6ba210-7964-459d-a6d2-0732097b0fa0\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-0eb76ba5-785cb7b9b6-c4g2f"
Sep 11 12:04:25.708108 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:25.708087    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gd66x\" (UniqueName: \"kubernetes.io/projected/2d6ba210-7964-459d-a6d2-0732097b0fa0-kube-api-access-gd66x\") pod \"payload-pre-processing-e2e-derive-0eb76ba5-785cb7b9b6-c4g2f\" (UID: \"2d6ba210-7964-459d-a6d2-0732097b0fa0\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-0eb76ba5-785cb7b9b6-c4g2f"
Sep 11 12:04:25.717607 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:25.717500    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-0eb76ba5-d9d447497-hc85x"]
Sep 11 12:04:25.810072 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:25.808991    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d6ba210-7964-459d-a6d2-0732097b0fa0-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-0eb76ba5-785cb7b9b6-c4g2f\" (UID: \"2d6ba210-7964-459d-a6d2-0732097b0fa0\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-0eb76ba5-785cb7b9b6-c4g2f"
Sep 11 12:04:25.810072 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:25.809046    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gd66x\" (UniqueName: \"kubernetes.io/projected/2d6ba210-7964-459d-a6d2-0732097b0fa0-kube-api-access-gd66x\") pod \"payload-pre-processing-e2e-derive-0eb76ba5-785cb7b9b6-c4g2f\" (UID: \"2d6ba210-7964-459d-a6d2-0732097b0fa0\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-0eb76ba5-785cb7b9b6-c4g2f"
Sep 11 12:04:25.810072 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:25.809157    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d8f10d4-ffec-4eca-9037-4958de3cfe2c-plugins-config-volume\") pod \"payload-processing-e2e-derive-0eb76ba5-d9d447497-hc85x\" (UID: \"9d8f10d4-ffec-4eca-9037-4958de3cfe2c\") " pod="openshift-ingress/payload-processing-e2e-derive-0eb76ba5-d9d447497-hc85x"
Sep 11 12:04:25.810072 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:25.809200    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkcrf\" (UniqueName: \"kubernetes.io/projected/9d8f10d4-ffec-4eca-9037-4958de3cfe2c-kube-api-access-pkcrf\") pod \"payload-processing-e2e-derive-0eb76ba5-d9d447497-hc85x\" (UID: \"9d8f10d4-ffec-4eca-9037-4958de3cfe2c\") " pod="openshift-ingress/payload-processing-e2e-derive-0eb76ba5-d9d447497-hc85x"
Sep 11 12:04:25.810604 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:25.810518    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d8f10d4-ffec-4eca-9037-4958de3cfe2c-plugins-config-volume\") pod \"payload-processing-e2e-derive-0eb76ba5-d9d447497-hc85x\" (UID: \"9d8f10d4-ffec-4eca-9037-4958de3cfe2c\") " pod="openshift-ingress/payload-processing-e2e-derive-0eb76ba5-d9d447497-hc85x"
Sep 11 12:04:25.811160 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:25.811135    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d6ba210-7964-459d-a6d2-0732097b0fa0-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-0eb76ba5-785cb7b9b6-c4g2f\" (UID: \"2d6ba210-7964-459d-a6d2-0732097b0fa0\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-0eb76ba5-785cb7b9b6-c4g2f"
Sep 11 12:04:25.819798 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:25.819723    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkcrf\" (UniqueName: \"kubernetes.io/projected/9d8f10d4-ffec-4eca-9037-4958de3cfe2c-kube-api-access-pkcrf\") pod \"payload-processing-e2e-derive-0eb76ba5-d9d447497-hc85x\" (UID: \"9d8f10d4-ffec-4eca-9037-4958de3cfe2c\") " pod="openshift-ingress/payload-processing-e2e-derive-0eb76ba5-d9d447497-hc85x"
Sep 11 12:04:25.822238 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:25.822205    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gd66x\" (UniqueName: \"kubernetes.io/projected/2d6ba210-7964-459d-a6d2-0732097b0fa0-kube-api-access-gd66x\") pod \"payload-pre-processing-e2e-derive-0eb76ba5-785cb7b9b6-c4g2f\" (UID: \"2d6ba210-7964-459d-a6d2-0732097b0fa0\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-0eb76ba5-785cb7b9b6-c4g2f"
Sep 11 12:04:25.986361 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:25.986322    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-0eb76ba5-785cb7b9b6-c4g2f"
Sep 11 12:04:26.007546 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:26.007515    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-0eb76ba5-d9d447497-hc85x"
Sep 11 12:04:26.268478 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:26.266470    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-0eb76ba5-785cb7b9b6-c4g2f"]
Sep 11 12:04:26.303611 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:26.303583    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-0eb76ba5-d9d447497-hc85x"]
Sep 11 12:04:26.307595 ip-10-0-142-145 kubenswrapper[2579]: W0911 12:04:26.307562    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d8f10d4_ffec_4eca_9037_4958de3cfe2c.slice/crio-22c1bb8c4637605e7efcad78a371b6d0b7d492216f90126bae2200547d2ba631 WatchSource:0}: Error finding container 22c1bb8c4637605e7efcad78a371b6d0b7d492216f90126bae2200547d2ba631: Status 404 returned error can't find the container with id 22c1bb8c4637605e7efcad78a371b6d0b7d492216f90126bae2200547d2ba631
Sep 11 12:04:27.124840 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:27.124794    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:04:27.205688 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:27.205651    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:27.257323 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:27.257287    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-0eb76ba5-d9d447497-hc85x" event={"ID":"9d8f10d4-ffec-4eca-9037-4958de3cfe2c","Type":"ContainerStarted","Data":"16e6ed65b5bd1ea6bdc989c079952a4eeb00f51c21b7386eadd503041b053dde"}
Sep 11 12:04:27.257552 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:27.257353    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-0eb76ba5-d9d447497-hc85x"
Sep 11 12:04:27.257552 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:27.257378    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-0eb76ba5-d9d447497-hc85x" event={"ID":"9d8f10d4-ffec-4eca-9037-4958de3cfe2c","Type":"ContainerStarted","Data":"22c1bb8c4637605e7efcad78a371b6d0b7d492216f90126bae2200547d2ba631"}
Sep 11 12:04:27.258886 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:27.258861    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-0eb76ba5-785cb7b9b6-c4g2f" event={"ID":"2d6ba210-7964-459d-a6d2-0732097b0fa0","Type":"ContainerStarted","Data":"735d051c2475cd70a0e4ccaf1c5fcd6ea97cd1fd263ffb6fb910da6ff476fd8e"}
Sep 11 12:04:27.258950 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:27.258915    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-0eb76ba5-785cb7b9b6-c4g2f" event={"ID":"2d6ba210-7964-459d-a6d2-0732097b0fa0","Type":"ContainerStarted","Data":"0e4775c8558b48407fb02336d60c28515bed45f522ab9e6017eb66f7c3d7fefc"}
Sep 11 12:04:27.258950 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:27.258939    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-0eb76ba5-785cb7b9b6-c4g2f"
Sep 11 12:04:27.280250 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:27.280191    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-0eb76ba5-d9d447497-hc85x" podStartSLOduration=2.280172885 podStartE2EDuration="2.280172885s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:04:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:04:27.279411296 +0000 UTC m=+971.176853046" watchObservedRunningTime="2026-09-11 12:04:27.280172885 +0000 UTC m=+971.177614634"
Sep 11 12:04:27.299892 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:27.299835    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-0eb76ba5-785cb7b9b6-c4g2f" podStartSLOduration=2.299817021 podStartE2EDuration="2.299817021s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:04:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:04:27.296478359 +0000 UTC m=+971.193920114" watchObservedRunningTime="2026-09-11 12:04:27.299817021 +0000 UTC m=+971.197258766"
Sep 11 12:04:36.529158 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:36.529115    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:38.266309 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:38.266275    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-0eb76ba5-785cb7b9b6-c4g2f"
Sep 11 12:04:38.266805 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:38.266469    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-0eb76ba5-d9d447497-hc85x"
Sep 11 12:04:43.176082 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:43.175985    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:04:47.373079 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:47.373044    2579 generic.go:356] "Generic (PLEG): container finished" podID="fca7bbc3-b185-48d3-bc1d-91a127f33876" containerID="67ca1480363e7556fee8bd4797abb6da52c240b80602ef30aefa2e4b4325146c" exitCode=0
Sep 11 12:04:47.373537 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:47.373111    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268" event={"ID":"fca7bbc3-b185-48d3-bc1d-91a127f33876","Type":"ContainerDied","Data":"67ca1480363e7556fee8bd4797abb6da52c240b80602ef30aefa2e4b4325146c"}
Sep 11 12:04:47.373589 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:47.373561    2579 scope.go:117] "RemoveContainer" containerID="67ca1480363e7556fee8bd4797abb6da52c240b80602ef30aefa2e4b4325146c"
Sep 11 12:04:47.374688 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:47.374662    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 12:04:47.374755 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:47.374732    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 12:04:47.374792 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:47.374776    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 12:04:48.381151 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:48.381114    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268" event={"ID":"fca7bbc3-b185-48d3-bc1d-91a127f33876","Type":"ContainerStarted","Data":"0b3d8c40e4925e54fc5da897884d60ca72624cecbb866f6e2a9d6372d676ef62"}
Sep 11 12:04:48.789513 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:48.789424    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268"
Sep 11 12:04:48.789655 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:48.789630    2579 patch_prober.go:28] interesting pod/e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Sep 11 12:04:48.789708 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:48.789685    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268" podUID="fca7bbc3-b185-48d3-bc1d-91a127f33876" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Sep 11 12:04:49.789626 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:49.789588    2579 patch_prober.go:28] interesting pod/e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Sep 11 12:04:49.790244 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:49.790209    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268" podUID="fca7bbc3-b185-48d3-bc1d-91a127f33876" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Sep 11 12:04:50.789967 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:50.789921    2579 patch_prober.go:28] interesting pod/e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Sep 11 12:04:50.790494 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:50.789998    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268" podUID="fca7bbc3-b185-48d3-bc1d-91a127f33876" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Sep 11 12:04:51.789090 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:51.789048    2579 patch_prober.go:28] interesting pod/e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Sep 11 12:04:51.789429 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:51.789371    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268" podUID="fca7bbc3-b185-48d3-bc1d-91a127f33876" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Sep 11 12:04:52.789115 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:52.789067    2579 patch_prober.go:28] interesting pod/e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Sep 11 12:04:52.789654 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:52.789138    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268" podUID="fca7bbc3-b185-48d3-bc1d-91a127f33876" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Sep 11 12:04:53.796415 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:53.788813    2579 patch_prober.go:28] interesting pod/e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Sep 11 12:04:53.796415 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:53.788889    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268" podUID="fca7bbc3-b185-48d3-bc1d-91a127f33876" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Sep 11 12:04:54.238474 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:54.238437    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268"
Sep 11 12:04:54.794304 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:54.794270    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268"
Sep 11 12:04:54.795369 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:54.795349    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268"
Sep 11 12:04:58.034520 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:04:58.034478    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:05:02.186338 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:02.186303    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:05:04.926982 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:04.926944    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:05:05.171220 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:05.171188    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:05:05.566689 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:05.566653    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:05:05.867725 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:05.867696    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:05:05.963214 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:05.963179    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-0eb76ba5-d9d447497-hc85x"]
Sep 11 12:05:05.963666 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:05.963448    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-0eb76ba5-d9d447497-hc85x" podUID="9d8f10d4-ffec-4eca-9037-4958de3cfe2c" containerName="payload-processing" containerID="cri-o://16e6ed65b5bd1ea6bdc989c079952a4eeb00f51c21b7386eadd503041b053dde" gracePeriod=30
Sep 11 12:05:05.968496 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:05.968470    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:05:05.975066 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:05.975040    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-0eb76ba5-785cb7b9b6-c4g2f"]
Sep 11 12:05:05.975329 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:05.975306    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-0eb76ba5-785cb7b9b6-c4g2f" podUID="2d6ba210-7964-459d-a6d2-0732097b0fa0" containerName="payload-pre-processing" containerID="cri-o://735d051c2475cd70a0e4ccaf1c5fcd6ea97cd1fd263ffb6fb910da6ff476fd8e" gracePeriod=30
Sep 11 12:05:06.087297 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:06.087229    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:05:06.203755 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:06.203719    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:05:06.255924 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:06.255901    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-0eb76ba5-d9d447497-hc85x"
Sep 11 12:05:06.258614 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:06.258592    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-0eb76ba5-785cb7b9b6-c4g2f"
Sep 11 12:05:06.290467 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:06.290424    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:05:06.327677 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:06.327621    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d8f10d4-ffec-4eca-9037-4958de3cfe2c-plugins-config-volume\") pod \"9d8f10d4-ffec-4eca-9037-4958de3cfe2c\" (UID: \"9d8f10d4-ffec-4eca-9037-4958de3cfe2c\") "
Sep 11 12:05:06.327921 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:06.327790    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gd66x\" (UniqueName: \"kubernetes.io/projected/2d6ba210-7964-459d-a6d2-0732097b0fa0-kube-api-access-gd66x\") pod \"2d6ba210-7964-459d-a6d2-0732097b0fa0\" (UID: \"2d6ba210-7964-459d-a6d2-0732097b0fa0\") "
Sep 11 12:05:06.327921 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:06.327865    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pkcrf\" (UniqueName: \"kubernetes.io/projected/9d8f10d4-ffec-4eca-9037-4958de3cfe2c-kube-api-access-pkcrf\") pod \"9d8f10d4-ffec-4eca-9037-4958de3cfe2c\" (UID: \"9d8f10d4-ffec-4eca-9037-4958de3cfe2c\") "
Sep 11 12:05:06.328113 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:06.327923    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d6ba210-7964-459d-a6d2-0732097b0fa0-plugins-config-volume\") pod \"2d6ba210-7964-459d-a6d2-0732097b0fa0\" (UID: \"2d6ba210-7964-459d-a6d2-0732097b0fa0\") "
Sep 11 12:05:06.328319 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:06.328269    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d8f10d4-ffec-4eca-9037-4958de3cfe2c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9d8f10d4-ffec-4eca-9037-4958de3cfe2c" (UID: "9d8f10d4-ffec-4eca-9037-4958de3cfe2c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:05:06.330442 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:06.330377    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d6ba210-7964-459d-a6d2-0732097b0fa0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2d6ba210-7964-459d-a6d2-0732097b0fa0" (UID: "2d6ba210-7964-459d-a6d2-0732097b0fa0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:05:06.332235 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:06.332204    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d6ba210-7964-459d-a6d2-0732097b0fa0-kube-api-access-gd66x" (OuterVolumeSpecName: "kube-api-access-gd66x") pod "2d6ba210-7964-459d-a6d2-0732097b0fa0" (UID: "2d6ba210-7964-459d-a6d2-0732097b0fa0"). InnerVolumeSpecName "kube-api-access-gd66x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:05:06.332366 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:06.332342    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d8f10d4-ffec-4eca-9037-4958de3cfe2c-kube-api-access-pkcrf" (OuterVolumeSpecName: "kube-api-access-pkcrf") pod "9d8f10d4-ffec-4eca-9037-4958de3cfe2c" (UID: "9d8f10d4-ffec-4eca-9037-4958de3cfe2c"). InnerVolumeSpecName "kube-api-access-pkcrf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:05:06.370039 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:06.370001    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268"]
Sep 11 12:05:06.370254 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:06.370231    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268" podUID="fca7bbc3-b185-48d3-bc1d-91a127f33876" containerName="istio-proxy" containerID="cri-o://0b3d8c40e4925e54fc5da897884d60ca72624cecbb866f6e2a9d6372d676ef62" gracePeriod=30
Sep 11 12:05:06.409626 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:06.409537    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:05:06.431176 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:06.431137    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gd66x\" (UniqueName: \"kubernetes.io/projected/2d6ba210-7964-459d-a6d2-0732097b0fa0-kube-api-access-gd66x\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:05:06.431176 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:06.431177    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pkcrf\" (UniqueName: \"kubernetes.io/projected/9d8f10d4-ffec-4eca-9037-4958de3cfe2c-kube-api-access-pkcrf\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:05:06.431383 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:06.431195    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d6ba210-7964-459d-a6d2-0732097b0fa0-plugins-config-volume\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:05:06.431383 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:06.431213    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d8f10d4-ffec-4eca-9037-4958de3cfe2c-plugins-config-volume\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:05:06.490957 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:06.490925    2579 generic.go:356] "Generic (PLEG): container finished" podID="2d6ba210-7964-459d-a6d2-0732097b0fa0" containerID="735d051c2475cd70a0e4ccaf1c5fcd6ea97cd1fd263ffb6fb910da6ff476fd8e" exitCode=0
Sep 11 12:05:06.491119 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:06.490969    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-0eb76ba5-785cb7b9b6-c4g2f"
Sep 11 12:05:06.491119 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:06.491002    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-0eb76ba5-785cb7b9b6-c4g2f" event={"ID":"2d6ba210-7964-459d-a6d2-0732097b0fa0","Type":"ContainerDied","Data":"735d051c2475cd70a0e4ccaf1c5fcd6ea97cd1fd263ffb6fb910da6ff476fd8e"}
Sep 11 12:05:06.491119 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:06.491057    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-0eb76ba5-785cb7b9b6-c4g2f" event={"ID":"2d6ba210-7964-459d-a6d2-0732097b0fa0","Type":"ContainerDied","Data":"0e4775c8558b48407fb02336d60c28515bed45f522ab9e6017eb66f7c3d7fefc"}
Sep 11 12:05:06.491119 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:06.491085    2579 scope.go:117] "RemoveContainer" containerID="735d051c2475cd70a0e4ccaf1c5fcd6ea97cd1fd263ffb6fb910da6ff476fd8e"
Sep 11 12:05:06.492703 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:06.492675    2579 generic.go:356] "Generic (PLEG): container finished" podID="9d8f10d4-ffec-4eca-9037-4958de3cfe2c" containerID="16e6ed65b5bd1ea6bdc989c079952a4eeb00f51c21b7386eadd503041b053dde" exitCode=0
Sep 11 12:05:06.492846 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:06.492714    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-0eb76ba5-d9d447497-hc85x" event={"ID":"9d8f10d4-ffec-4eca-9037-4958de3cfe2c","Type":"ContainerDied","Data":"16e6ed65b5bd1ea6bdc989c079952a4eeb00f51c21b7386eadd503041b053dde"}
Sep 11 12:05:06.492846 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:06.492727    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-0eb76ba5-d9d447497-hc85x"
Sep 11 12:05:06.492846 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:06.492762    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-0eb76ba5-d9d447497-hc85x" event={"ID":"9d8f10d4-ffec-4eca-9037-4958de3cfe2c","Type":"ContainerDied","Data":"22c1bb8c4637605e7efcad78a371b6d0b7d492216f90126bae2200547d2ba631"}
Sep 11 12:05:06.500693 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:06.500671    2579 scope.go:117] "RemoveContainer" containerID="735d051c2475cd70a0e4ccaf1c5fcd6ea97cd1fd263ffb6fb910da6ff476fd8e"
Sep 11 12:05:06.501154 ip-10-0-142-145 kubenswrapper[2579]: E0911 12:05:06.501128    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"735d051c2475cd70a0e4ccaf1c5fcd6ea97cd1fd263ffb6fb910da6ff476fd8e\": container with ID starting with 735d051c2475cd70a0e4ccaf1c5fcd6ea97cd1fd263ffb6fb910da6ff476fd8e not found: ID does not exist" containerID="735d051c2475cd70a0e4ccaf1c5fcd6ea97cd1fd263ffb6fb910da6ff476fd8e"
Sep 11 12:05:06.501259 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:06.501160    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"735d051c2475cd70a0e4ccaf1c5fcd6ea97cd1fd263ffb6fb910da6ff476fd8e"} err="failed to get container status \"735d051c2475cd70a0e4ccaf1c5fcd6ea97cd1fd263ffb6fb910da6ff476fd8e\": rpc error: code = NotFound desc = could not find container \"735d051c2475cd70a0e4ccaf1c5fcd6ea97cd1fd263ffb6fb910da6ff476fd8e\": container with ID starting with 735d051c2475cd70a0e4ccaf1c5fcd6ea97cd1fd263ffb6fb910da6ff476fd8e not found: ID does not exist"
Sep 11 12:05:06.501259 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:06.501182    2579 scope.go:117] "RemoveContainer" containerID="16e6ed65b5bd1ea6bdc989c079952a4eeb00f51c21b7386eadd503041b053dde"
Sep 11 12:05:06.515830 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:06.515799    2579 scope.go:117] "RemoveContainer" containerID="16e6ed65b5bd1ea6bdc989c079952a4eeb00f51c21b7386eadd503041b053dde"
Sep 11 12:05:06.528600 ip-10-0-142-145 kubenswrapper[2579]: E0911 12:05:06.528554    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"16e6ed65b5bd1ea6bdc989c079952a4eeb00f51c21b7386eadd503041b053dde\": container with ID starting with 16e6ed65b5bd1ea6bdc989c079952a4eeb00f51c21b7386eadd503041b053dde not found: ID does not exist" containerID="16e6ed65b5bd1ea6bdc989c079952a4eeb00f51c21b7386eadd503041b053dde"
Sep 11 12:05:06.528822 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:06.528605    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16e6ed65b5bd1ea6bdc989c079952a4eeb00f51c21b7386eadd503041b053dde"} err="failed to get container status \"16e6ed65b5bd1ea6bdc989c079952a4eeb00f51c21b7386eadd503041b053dde\": rpc error: code = NotFound desc = could not find container \"16e6ed65b5bd1ea6bdc989c079952a4eeb00f51c21b7386eadd503041b053dde\": container with ID starting with 16e6ed65b5bd1ea6bdc989c079952a4eeb00f51c21b7386eadd503041b053dde not found: ID does not exist"
Sep 11 12:05:06.533490 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:06.533454    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-0eb76ba5-785cb7b9b6-c4g2f"]
Sep 11 12:05:06.537890 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:06.537862    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-0eb76ba5-785cb7b9b6-c4g2f"]
Sep 11 12:05:06.550104 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:06.550075    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-0eb76ba5-d9d447497-hc85x"]
Sep 11 12:05:06.554801 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:06.554774    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-0eb76ba5-d9d447497-hc85x"]
Sep 11 12:05:06.576181 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:06.576148    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d6ba210-7964-459d-a6d2-0732097b0fa0" path="/var/lib/kubelet/pods/2d6ba210-7964-459d-a6d2-0732097b0fa0/volumes"
Sep 11 12:05:06.576562 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:06.576547    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d8f10d4-ffec-4eca-9037-4958de3cfe2c" path="/var/lib/kubelet/pods/9d8f10d4-ffec-4eca-9037-4958de3cfe2c/volumes"
Sep 11 12:05:07.603317 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:07.603275    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:05:07.703197 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:07.703160    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:05:08.641004 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:08.640973    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:05:08.739702 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:08.739666    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:05:09.156085 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:09.156049    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:05:09.261459 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:09.261418    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:05:09.796993 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:09.796957    2579 patch_prober.go:28] interesting pod/e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
Sep 11 12:05:09.797368 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:09.797024    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268" podUID="fca7bbc3-b185-48d3-bc1d-91a127f33876" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 12:05:10.594086 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:10.594042    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:05:10.696672 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:10.696632    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:05:11.397915 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:11.397879    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:05:11.553409 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:11.553362    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:05:11.617910 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:11.617872    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-69b1ea-7bf7698f94-wg896"]
Sep 11 12:05:11.618556 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:11.618535    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d8f10d4-ffec-4eca-9037-4958de3cfe2c" containerName="payload-processing"
Sep 11 12:05:11.618609 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:11.618561    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d8f10d4-ffec-4eca-9037-4958de3cfe2c" containerName="payload-processing"
Sep 11 12:05:11.618659 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:11.618609    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d6ba210-7964-459d-a6d2-0732097b0fa0" containerName="payload-pre-processing"
Sep 11 12:05:11.618659 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:11.618626    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d6ba210-7964-459d-a6d2-0732097b0fa0" containerName="payload-pre-processing"
Sep 11 12:05:11.618737 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:11.618725    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d6ba210-7964-459d-a6d2-0732097b0fa0" containerName="payload-pre-processing"
Sep 11 12:05:11.618775 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:11.618741    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d8f10d4-ffec-4eca-9037-4958de3cfe2c" containerName="payload-processing"
Sep 11 12:05:11.621275 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:11.621257    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-69b1ea-7bf7698f94-wg896"
Sep 11 12:05:11.624952 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:11.624933    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-69b1ea\""
Sep 11 12:05:11.625574 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:11.625559    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-69b1ea-dockercfg-qjpsr\""
Sep 11 12:05:11.636093 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:11.636064    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-69b1ea-7bf7698f94-wg896"]
Sep 11 12:05:11.683878 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:11.683805    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwtlj\" (UniqueName: \"kubernetes.io/projected/163132be-897b-4e52-aa5d-e71848911b96-kube-api-access-vwtlj\") pod \"payload-pre-processing-e2e-explicit-69b1ea-7bf7698f94-wg896\" (UID: \"163132be-897b-4e52-aa5d-e71848911b96\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-69b1ea-7bf7698f94-wg896"
Sep 11 12:05:11.684017 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:11.683929    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/163132be-897b-4e52-aa5d-e71848911b96-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-69b1ea-7bf7698f94-wg896\" (UID: \"163132be-897b-4e52-aa5d-e71848911b96\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-69b1ea-7bf7698f94-wg896"
Sep 11 12:05:11.749653 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:11.749625    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268"
Sep 11 12:05:11.784541 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:11.784502    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fca7bbc3-b185-48d3-bc1d-91a127f33876-credential-socket\") pod \"fca7bbc3-b185-48d3-bc1d-91a127f33876\" (UID: \"fca7bbc3-b185-48d3-bc1d-91a127f33876\") "
Sep 11 12:05:11.784735 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:11.784554    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6g997\" (UniqueName: \"kubernetes.io/projected/fca7bbc3-b185-48d3-bc1d-91a127f33876-kube-api-access-6g997\") pod \"fca7bbc3-b185-48d3-bc1d-91a127f33876\" (UID: \"fca7bbc3-b185-48d3-bc1d-91a127f33876\") "
Sep 11 12:05:11.784735 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:11.784582    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fca7bbc3-b185-48d3-bc1d-91a127f33876-istiod-ca-cert\") pod \"fca7bbc3-b185-48d3-bc1d-91a127f33876\" (UID: \"fca7bbc3-b185-48d3-bc1d-91a127f33876\") "
Sep 11 12:05:11.784735 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:11.784617    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fca7bbc3-b185-48d3-bc1d-91a127f33876-istio-podinfo\") pod \"fca7bbc3-b185-48d3-bc1d-91a127f33876\" (UID: \"fca7bbc3-b185-48d3-bc1d-91a127f33876\") "
Sep 11 12:05:11.784735 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:11.784669    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fca7bbc3-b185-48d3-bc1d-91a127f33876-workload-certs\") pod \"fca7bbc3-b185-48d3-bc1d-91a127f33876\" (UID: \"fca7bbc3-b185-48d3-bc1d-91a127f33876\") "
Sep 11 12:05:11.784958 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:11.784825    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fca7bbc3-b185-48d3-bc1d-91a127f33876-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fca7bbc3-b185-48d3-bc1d-91a127f33876" (UID: "fca7bbc3-b185-48d3-bc1d-91a127f33876"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:05:11.784958 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:11.784836    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fca7bbc3-b185-48d3-bc1d-91a127f33876-istio-data\") pod \"fca7bbc3-b185-48d3-bc1d-91a127f33876\" (UID: \"fca7bbc3-b185-48d3-bc1d-91a127f33876\") "
Sep 11 12:05:11.785098 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:11.785069    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fca7bbc3-b185-48d3-bc1d-91a127f33876-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fca7bbc3-b185-48d3-bc1d-91a127f33876" (UID: "fca7bbc3-b185-48d3-bc1d-91a127f33876"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:05:11.785162 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:11.785095    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fca7bbc3-b185-48d3-bc1d-91a127f33876-istio-data" (OuterVolumeSpecName: "istio-data") pod "fca7bbc3-b185-48d3-bc1d-91a127f33876" (UID: "fca7bbc3-b185-48d3-bc1d-91a127f33876"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:05:11.785210 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:11.785163    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fca7bbc3-b185-48d3-bc1d-91a127f33876-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fca7bbc3-b185-48d3-bc1d-91a127f33876" (UID: "fca7bbc3-b185-48d3-bc1d-91a127f33876"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:05:11.785315 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:11.785269    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fca7bbc3-b185-48d3-bc1d-91a127f33876-istio-envoy\") pod \"fca7bbc3-b185-48d3-bc1d-91a127f33876\" (UID: \"fca7bbc3-b185-48d3-bc1d-91a127f33876\") "
Sep 11 12:05:11.785368 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:11.785323    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fca7bbc3-b185-48d3-bc1d-91a127f33876-workload-socket\") pod \"fca7bbc3-b185-48d3-bc1d-91a127f33876\" (UID: \"fca7bbc3-b185-48d3-bc1d-91a127f33876\") "
Sep 11 12:05:11.785471 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:11.785378    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fca7bbc3-b185-48d3-bc1d-91a127f33876-istio-token\") pod \"fca7bbc3-b185-48d3-bc1d-91a127f33876\" (UID: \"fca7bbc3-b185-48d3-bc1d-91a127f33876\") "
Sep 11 12:05:11.785669 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:11.785643    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwtlj\" (UniqueName: \"kubernetes.io/projected/163132be-897b-4e52-aa5d-e71848911b96-kube-api-access-vwtlj\") pod \"payload-pre-processing-e2e-explicit-69b1ea-7bf7698f94-wg896\" (UID: \"163132be-897b-4e52-aa5d-e71848911b96\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-69b1ea-7bf7698f94-wg896"
Sep 11 12:05:11.785964 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:11.785939    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fca7bbc3-b185-48d3-bc1d-91a127f33876-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fca7bbc3-b185-48d3-bc1d-91a127f33876" (UID: "fca7bbc3-b185-48d3-bc1d-91a127f33876"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:05:11.786071 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:11.785943    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/163132be-897b-4e52-aa5d-e71848911b96-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-69b1ea-7bf7698f94-wg896\" (UID: \"163132be-897b-4e52-aa5d-e71848911b96\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-69b1ea-7bf7698f94-wg896"
Sep 11 12:05:11.786136 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:11.786111    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fca7bbc3-b185-48d3-bc1d-91a127f33876-workload-socket\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:05:11.786136 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:11.786130    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fca7bbc3-b185-48d3-bc1d-91a127f33876-credential-socket\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:05:11.786243 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:11.786143    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fca7bbc3-b185-48d3-bc1d-91a127f33876-istiod-ca-cert\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:05:11.786243 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:11.786157    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fca7bbc3-b185-48d3-bc1d-91a127f33876-workload-certs\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:05:11.786243 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:11.786171    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fca7bbc3-b185-48d3-bc1d-91a127f33876-istio-data\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:05:11.787206 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:11.787172    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fca7bbc3-b185-48d3-bc1d-91a127f33876-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fca7bbc3-b185-48d3-bc1d-91a127f33876" (UID: "fca7bbc3-b185-48d3-bc1d-91a127f33876"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 12:05:11.787408 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:11.787363    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fca7bbc3-b185-48d3-bc1d-91a127f33876-kube-api-access-6g997" (OuterVolumeSpecName: "kube-api-access-6g997") pod "fca7bbc3-b185-48d3-bc1d-91a127f33876" (UID: "fca7bbc3-b185-48d3-bc1d-91a127f33876"). InnerVolumeSpecName "kube-api-access-6g997". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:05:11.787542 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:11.787446    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/163132be-897b-4e52-aa5d-e71848911b96-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-69b1ea-7bf7698f94-wg896\" (UID: \"163132be-897b-4e52-aa5d-e71848911b96\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-69b1ea-7bf7698f94-wg896"
Sep 11 12:05:11.787542 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:11.787449    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fca7bbc3-b185-48d3-bc1d-91a127f33876-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fca7bbc3-b185-48d3-bc1d-91a127f33876" (UID: "fca7bbc3-b185-48d3-bc1d-91a127f33876"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:05:11.787719 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:11.787698    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fca7bbc3-b185-48d3-bc1d-91a127f33876-istio-token" (OuterVolumeSpecName: "istio-token") pod "fca7bbc3-b185-48d3-bc1d-91a127f33876" (UID: "fca7bbc3-b185-48d3-bc1d-91a127f33876"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:05:11.800447 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:11.800415    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwtlj\" (UniqueName: \"kubernetes.io/projected/163132be-897b-4e52-aa5d-e71848911b96-kube-api-access-vwtlj\") pod \"payload-pre-processing-e2e-explicit-69b1ea-7bf7698f94-wg896\" (UID: \"163132be-897b-4e52-aa5d-e71848911b96\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-69b1ea-7bf7698f94-wg896"
Sep 11 12:05:11.887075 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:11.887036    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fca7bbc3-b185-48d3-bc1d-91a127f33876-istio-envoy\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:05:11.887075 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:11.887071    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fca7bbc3-b185-48d3-bc1d-91a127f33876-istio-token\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:05:11.887075 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:11.887082    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6g997\" (UniqueName: \"kubernetes.io/projected/fca7bbc3-b185-48d3-bc1d-91a127f33876-kube-api-access-6g997\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:05:11.887314 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:11.887092    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fca7bbc3-b185-48d3-bc1d-91a127f33876-istio-podinfo\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:05:11.933214 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:11.933163    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-69b1ea-7bf7698f94-wg896"
Sep 11 12:05:12.219451 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:12.219342    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-69b1ea-7bf7698f94-wg896"]
Sep 11 12:05:12.222128 ip-10-0-142-145 kubenswrapper[2579]: W0911 12:05:12.222101    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod163132be_897b_4e52_aa5d_e71848911b96.slice/crio-42f738bcba9612a8d09b6ceb1afbf4704b98fa88d4a52f7ab2554643b450a36c WatchSource:0}: Error finding container 42f738bcba9612a8d09b6ceb1afbf4704b98fa88d4a52f7ab2554643b450a36c: Status 404 returned error can't find the container with id 42f738bcba9612a8d09b6ceb1afbf4704b98fa88d4a52f7ab2554643b450a36c
Sep 11 12:05:12.529786 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:12.529693    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-69b1ea-7bf7698f94-wg896" event={"ID":"163132be-897b-4e52-aa5d-e71848911b96","Type":"ContainerStarted","Data":"123d57604628f18093b7159985cd9b40d2553858104dbf21601c6f1f361632dd"}
Sep 11 12:05:12.529786 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:12.529752    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-69b1ea-7bf7698f94-wg896" event={"ID":"163132be-897b-4e52-aa5d-e71848911b96","Type":"ContainerStarted","Data":"42f738bcba9612a8d09b6ceb1afbf4704b98fa88d4a52f7ab2554643b450a36c"}
Sep 11 12:05:12.530240 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:12.529883    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-69b1ea-7bf7698f94-wg896"
Sep 11 12:05:12.531537 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:12.531510    2579 generic.go:356] "Generic (PLEG): container finished" podID="fca7bbc3-b185-48d3-bc1d-91a127f33876" containerID="0b3d8c40e4925e54fc5da897884d60ca72624cecbb866f6e2a9d6372d676ef62" exitCode=0
Sep 11 12:05:12.531694 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:12.531559    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268"
Sep 11 12:05:12.531694 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:12.531572    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268" event={"ID":"fca7bbc3-b185-48d3-bc1d-91a127f33876","Type":"ContainerDied","Data":"0b3d8c40e4925e54fc5da897884d60ca72624cecbb866f6e2a9d6372d676ef62"}
Sep 11 12:05:12.531694 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:12.531610    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268" event={"ID":"fca7bbc3-b185-48d3-bc1d-91a127f33876","Type":"ContainerDied","Data":"f4f8069f0bf7aa312b8e6f4889d3c215281b9fc7813e09c91e181e794d8479af"}
Sep 11 12:05:12.531694 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:12.531630    2579 scope.go:117] "RemoveContainer" containerID="0b3d8c40e4925e54fc5da897884d60ca72624cecbb866f6e2a9d6372d676ef62"
Sep 11 12:05:12.542133 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:12.542111    2579 scope.go:117] "RemoveContainer" containerID="67ca1480363e7556fee8bd4797abb6da52c240b80602ef30aefa2e4b4325146c"
Sep 11 12:05:12.555581 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:12.555433    2579 scope.go:117] "RemoveContainer" containerID="0b3d8c40e4925e54fc5da897884d60ca72624cecbb866f6e2a9d6372d676ef62"
Sep 11 12:05:12.556010 ip-10-0-142-145 kubenswrapper[2579]: E0911 12:05:12.555985    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b3d8c40e4925e54fc5da897884d60ca72624cecbb866f6e2a9d6372d676ef62\": container with ID starting with 0b3d8c40e4925e54fc5da897884d60ca72624cecbb866f6e2a9d6372d676ef62 not found: ID does not exist" containerID="0b3d8c40e4925e54fc5da897884d60ca72624cecbb866f6e2a9d6372d676ef62"
Sep 11 12:05:12.556149 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:12.556021    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b3d8c40e4925e54fc5da897884d60ca72624cecbb866f6e2a9d6372d676ef62"} err="failed to get container status \"0b3d8c40e4925e54fc5da897884d60ca72624cecbb866f6e2a9d6372d676ef62\": rpc error: code = NotFound desc = could not find container \"0b3d8c40e4925e54fc5da897884d60ca72624cecbb866f6e2a9d6372d676ef62\": container with ID starting with 0b3d8c40e4925e54fc5da897884d60ca72624cecbb866f6e2a9d6372d676ef62 not found: ID does not exist"
Sep 11 12:05:12.556149 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:12.556046    2579 scope.go:117] "RemoveContainer" containerID="67ca1480363e7556fee8bd4797abb6da52c240b80602ef30aefa2e4b4325146c"
Sep 11 12:05:12.556465 ip-10-0-142-145 kubenswrapper[2579]: E0911 12:05:12.556445    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67ca1480363e7556fee8bd4797abb6da52c240b80602ef30aefa2e4b4325146c\": container with ID starting with 67ca1480363e7556fee8bd4797abb6da52c240b80602ef30aefa2e4b4325146c not found: ID does not exist" containerID="67ca1480363e7556fee8bd4797abb6da52c240b80602ef30aefa2e4b4325146c"
Sep 11 12:05:12.556554 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:12.556476    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67ca1480363e7556fee8bd4797abb6da52c240b80602ef30aefa2e4b4325146c"} err="failed to get container status \"67ca1480363e7556fee8bd4797abb6da52c240b80602ef30aefa2e4b4325146c\": rpc error: code = NotFound desc = could not find container \"67ca1480363e7556fee8bd4797abb6da52c240b80602ef30aefa2e4b4325146c\": container with ID starting with 67ca1480363e7556fee8bd4797abb6da52c240b80602ef30aefa2e4b4325146c not found: ID does not exist"
Sep 11 12:05:12.556554 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:12.556503    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-explicit-69b1ea-7bf7698f94-wg896" podStartSLOduration=1.5564843430000002 podStartE2EDuration="1.556484343s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:05:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:05:12.552705339 +0000 UTC m=+1016.450147084" watchObservedRunningTime="2026-09-11 12:05:12.556484343 +0000 UTC m=+1016.453926089"
Sep 11 12:05:12.582034 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:12.581996    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268"]
Sep 11 12:05:12.589841 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:12.589811    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-0eb76ba5-openshift-default-c4fc88bdd-vm268"]
Sep 11 12:05:14.579407 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:14.579359    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fca7bbc3-b185-48d3-bc1d-91a127f33876" path="/var/lib/kubelet/pods/fca7bbc3-b185-48d3-bc1d-91a127f33876/volumes"
Sep 11 12:05:15.820934 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:15.820903    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:05:16.050297 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:16.050257    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:05:16.372685 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:16.372649    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:05:16.537787 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:16.537755    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:05:16.652094 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:16.652001    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:05:23.540660 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:23.540628    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-69b1ea-7bf7698f94-wg896"
Sep 11 12:05:30.251600 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:30.251565    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:05:34.079764 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:34.079722    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:05:36.769959 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:36.769927    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:05:42.526548 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:42.526513    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:05:42.627565 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:42.627526    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:05:48.457558 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:48.457519    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:05:49.148022 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:49.147988    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:05:49.267272 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:49.267229    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:05:49.448493 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:49.448368    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:05:50.933927 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:50.933892    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:05:51.028939 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:51.028907    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-69b1ea-7lr4j"]
Sep 11 12:05:51.029338 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:51.029326    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fca7bbc3-b185-48d3-bc1d-91a127f33876" containerName="istio-proxy"
Sep 11 12:05:51.029381 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:51.029339    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="fca7bbc3-b185-48d3-bc1d-91a127f33876" containerName="istio-proxy"
Sep 11 12:05:51.029381 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:51.029355    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fca7bbc3-b185-48d3-bc1d-91a127f33876" containerName="istio-proxy"
Sep 11 12:05:51.029381 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:51.029361    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="fca7bbc3-b185-48d3-bc1d-91a127f33876" containerName="istio-proxy"
Sep 11 12:05:51.029498 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:51.029468    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="fca7bbc3-b185-48d3-bc1d-91a127f33876" containerName="istio-proxy"
Sep 11 12:05:51.031870 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:51.031853    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-69b1ea-7lr4j"
Sep 11 12:05:51.034674 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:51.034651    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9xkpf\""
Sep 11 12:05:51.041888 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:51.041859    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-69b1ea-7lr4j"]
Sep 11 12:05:51.071710 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:51.071675    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d3fd160-8d07-4b01-9c87-343d8b6d35b8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-69b1ea-7lr4j\" (UID: \"4d3fd160-8d07-4b01-9c87-343d8b6d35b8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-69b1ea-7lr4j"
Sep 11 12:05:51.173110 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:51.173070    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d3fd160-8d07-4b01-9c87-343d8b6d35b8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-69b1ea-7lr4j\" (UID: \"4d3fd160-8d07-4b01-9c87-343d8b6d35b8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-69b1ea-7lr4j"
Sep 11 12:05:51.173817 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:51.173791    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d3fd160-8d07-4b01-9c87-343d8b6d35b8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-69b1ea-7lr4j\" (UID: \"4d3fd160-8d07-4b01-9c87-343d8b6d35b8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-69b1ea-7lr4j"
Sep 11 12:05:51.343642 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:51.343608    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-69b1ea-7lr4j"
Sep 11 12:05:51.622429 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:51.620713    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-69b1ea-7lr4j"]
Sep 11 12:05:51.625378 ip-10-0-142-145 kubenswrapper[2579]: W0911 12:05:51.625348    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d3fd160_8d07_4b01_9c87_343d8b6d35b8.slice/crio-64f4e9a6fbf63249667a82db012b78c10e44eca95d1ca192e77e999c0c32af65 WatchSource:0}: Error finding container 64f4e9a6fbf63249667a82db012b78c10e44eca95d1ca192e77e999c0c32af65: Status 404 returned error can't find the container with id 64f4e9a6fbf63249667a82db012b78c10e44eca95d1ca192e77e999c0c32af65
Sep 11 12:05:51.746104 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:51.746064    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-69b1ea-7lr4j" event={"ID":"4d3fd160-8d07-4b01-9c87-343d8b6d35b8","Type":"ContainerStarted","Data":"64f4e9a6fbf63249667a82db012b78c10e44eca95d1ca192e77e999c0c32af65"}
Sep 11 12:05:52.752871 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:52.752832    2579 generic.go:356] "Generic (PLEG): container finished" podID="4d3fd160-8d07-4b01-9c87-343d8b6d35b8" containerID="4fb76d47e3354f7a699294551c0c13eb42109f65312d06d28dae8f6f611efd69" exitCode=0
Sep 11 12:05:52.753287 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:52.752889    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-69b1ea-7lr4j" event={"ID":"4d3fd160-8d07-4b01-9c87-343d8b6d35b8","Type":"ContainerDied","Data":"4fb76d47e3354f7a699294551c0c13eb42109f65312d06d28dae8f6f611efd69"}
Sep 11 12:05:53.662075 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:53.662041    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:05:53.900168 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:53.900142    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-69b1ea-7lr4j"
Sep 11 12:05:54.001373 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:54.001298    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d3fd160-8d07-4b01-9c87-343d8b6d35b8-maas-api-service-ca\") pod \"4d3fd160-8d07-4b01-9c87-343d8b6d35b8\" (UID: \"4d3fd160-8d07-4b01-9c87-343d8b6d35b8\") "
Sep 11 12:05:54.001630 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:54.001609    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d3fd160-8d07-4b01-9c87-343d8b6d35b8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4d3fd160-8d07-4b01-9c87-343d8b6d35b8" (UID: "4d3fd160-8d07-4b01-9c87-343d8b6d35b8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:05:54.102781 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:54.102746    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d3fd160-8d07-4b01-9c87-343d8b6d35b8-maas-api-service-ca\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:05:54.765008 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:54.764979    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-69b1ea-7lr4j"
Sep 11 12:05:54.765220 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:54.765015    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-69b1ea-7lr4j" event={"ID":"4d3fd160-8d07-4b01-9c87-343d8b6d35b8","Type":"ContainerDied","Data":"64f4e9a6fbf63249667a82db012b78c10e44eca95d1ca192e77e999c0c32af65"}
Sep 11 12:05:54.765220 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:05:54.765073    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="64f4e9a6fbf63249667a82db012b78c10e44eca95d1ca192e77e999c0c32af65"
Sep 11 12:06:01.687719 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:01.687677    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:01.729135 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:01.729104    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-69b1ea-7bf7698f94-wg896"]
Sep 11 12:06:01.729432 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:01.729387    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-explicit-69b1ea-7bf7698f94-wg896" podUID="163132be-897b-4e52-aa5d-e71848911b96" containerName="payload-pre-processing" containerID="cri-o://123d57604628f18093b7159985cd9b40d2553858104dbf21601c6f1f361632dd" gracePeriod=30
Sep 11 12:06:01.763419 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:01.763372    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:01.840579 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:01.840526    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:01.901265 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:01.901229    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-69b1ea-7lr4j"]
Sep 11 12:06:01.904493 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:01.904463    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-69b1ea-7lr4j"]
Sep 11 12:06:01.911331 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:01.911303    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:01.986935 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:01.986904    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-69b1ea-7bf7698f94-wg896"
Sep 11 12:06:02.083363 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:02.083326    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vwtlj\" (UniqueName: \"kubernetes.io/projected/163132be-897b-4e52-aa5d-e71848911b96-kube-api-access-vwtlj\") pod \"163132be-897b-4e52-aa5d-e71848911b96\" (UID: \"163132be-897b-4e52-aa5d-e71848911b96\") "
Sep 11 12:06:02.083559 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:02.083434    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/163132be-897b-4e52-aa5d-e71848911b96-plugins-config-volume\") pod \"163132be-897b-4e52-aa5d-e71848911b96\" (UID: \"163132be-897b-4e52-aa5d-e71848911b96\") "
Sep 11 12:06:02.083867 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:02.083844    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/163132be-897b-4e52-aa5d-e71848911b96-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "163132be-897b-4e52-aa5d-e71848911b96" (UID: "163132be-897b-4e52-aa5d-e71848911b96"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:06:02.085459 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:02.085433    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/163132be-897b-4e52-aa5d-e71848911b96-kube-api-access-vwtlj" (OuterVolumeSpecName: "kube-api-access-vwtlj") pod "163132be-897b-4e52-aa5d-e71848911b96" (UID: "163132be-897b-4e52-aa5d-e71848911b96"). InnerVolumeSpecName "kube-api-access-vwtlj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:06:02.184779 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:02.184739    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vwtlj\" (UniqueName: \"kubernetes.io/projected/163132be-897b-4e52-aa5d-e71848911b96-kube-api-access-vwtlj\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:06:02.184779 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:02.184772    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/163132be-897b-4e52-aa5d-e71848911b96-plugins-config-volume\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:06:02.235173 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:02.235077    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:02.577191 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:02.577102    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d3fd160-8d07-4b01-9c87-343d8b6d35b8" path="/var/lib/kubelet/pods/4d3fd160-8d07-4b01-9c87-343d8b6d35b8/volumes"
Sep 11 12:06:02.817853 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:02.817811    2579 generic.go:356] "Generic (PLEG): container finished" podID="163132be-897b-4e52-aa5d-e71848911b96" containerID="123d57604628f18093b7159985cd9b40d2553858104dbf21601c6f1f361632dd" exitCode=0
Sep 11 12:06:02.817853 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:02.817855    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-69b1ea-7bf7698f94-wg896"
Sep 11 12:06:02.818423 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:02.817903    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-69b1ea-7bf7698f94-wg896" event={"ID":"163132be-897b-4e52-aa5d-e71848911b96","Type":"ContainerDied","Data":"123d57604628f18093b7159985cd9b40d2553858104dbf21601c6f1f361632dd"}
Sep 11 12:06:02.818423 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:02.817960    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-69b1ea-7bf7698f94-wg896" event={"ID":"163132be-897b-4e52-aa5d-e71848911b96","Type":"ContainerDied","Data":"42f738bcba9612a8d09b6ceb1afbf4704b98fa88d4a52f7ab2554643b450a36c"}
Sep 11 12:06:02.818423 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:02.817975    2579 scope.go:117] "RemoveContainer" containerID="123d57604628f18093b7159985cd9b40d2553858104dbf21601c6f1f361632dd"
Sep 11 12:06:02.827438 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:02.827415    2579 scope.go:117] "RemoveContainer" containerID="123d57604628f18093b7159985cd9b40d2553858104dbf21601c6f1f361632dd"
Sep 11 12:06:02.827762 ip-10-0-142-145 kubenswrapper[2579]: E0911 12:06:02.827741    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"123d57604628f18093b7159985cd9b40d2553858104dbf21601c6f1f361632dd\": container with ID starting with 123d57604628f18093b7159985cd9b40d2553858104dbf21601c6f1f361632dd not found: ID does not exist" containerID="123d57604628f18093b7159985cd9b40d2553858104dbf21601c6f1f361632dd"
Sep 11 12:06:02.827835 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:02.827775    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"123d57604628f18093b7159985cd9b40d2553858104dbf21601c6f1f361632dd"} err="failed to get container status \"123d57604628f18093b7159985cd9b40d2553858104dbf21601c6f1f361632dd\": rpc error: code = NotFound desc = could not find container \"123d57604628f18093b7159985cd9b40d2553858104dbf21601c6f1f361632dd\": container with ID starting with 123d57604628f18093b7159985cd9b40d2553858104dbf21601c6f1f361632dd not found: ID does not exist"
Sep 11 12:06:02.836072 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:02.836038    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-69b1ea-7bf7698f94-wg896"]
Sep 11 12:06:02.840110 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:02.840080    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-69b1ea-7bf7698f94-wg896"]
Sep 11 12:06:04.578435 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:04.578382    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="163132be-897b-4e52-aa5d-e71848911b96" path="/var/lib/kubelet/pods/163132be-897b-4e52-aa5d-e71848911b96/volumes"
Sep 11 12:06:05.385141 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:05.385104    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:07.315693 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:07.315659    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:07.386061 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:07.386033    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:09.298262 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:09.298168    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:09.399295 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:09.399265    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:09.654644 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:09.654598    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:10.046947 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:10.046864    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:10.339212 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:10.339136    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6"]
Sep 11 12:06:10.339595 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:10.339580    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d3fd160-8d07-4b01-9c87-343d8b6d35b8" containerName="revoke-keys"
Sep 11 12:06:10.339643 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:10.339596    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d3fd160-8d07-4b01-9c87-343d8b6d35b8" containerName="revoke-keys"
Sep 11 12:06:10.339643 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:10.339608    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="163132be-897b-4e52-aa5d-e71848911b96" containerName="payload-pre-processing"
Sep 11 12:06:10.339643 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:10.339614    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="163132be-897b-4e52-aa5d-e71848911b96" containerName="payload-pre-processing"
Sep 11 12:06:10.339739 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:10.339683    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="fca7bbc3-b185-48d3-bc1d-91a127f33876" containerName="istio-proxy"
Sep 11 12:06:10.339739 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:10.339690    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="163132be-897b-4e52-aa5d-e71848911b96" containerName="payload-pre-processing"
Sep 11 12:06:10.339739 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:10.339698    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d3fd160-8d07-4b01-9c87-343d8b6d35b8" containerName="revoke-keys"
Sep 11 12:06:10.342818 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:10.342793    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6"
Sep 11 12:06:10.345730 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:10.345706    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-noten-4a674b-openshift-default-dockercfg-8jpk8\""
Sep 11 12:06:10.362385 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:10.362357    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6"]
Sep 11 12:06:10.393885 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:10.393844    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:10.463828 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:10.463789    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f757480-6967-4af8-8517-9110e0956da4-istio-envoy\") pod \"e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6\" (UID: \"8f757480-6967-4af8-8517-9110e0956da4\") " pod="openshift-ingress/e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6"
Sep 11 12:06:10.463828 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:10.463833    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f757480-6967-4af8-8517-9110e0956da4-istiod-ca-cert\") pod \"e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6\" (UID: \"8f757480-6967-4af8-8517-9110e0956da4\") " pod="openshift-ingress/e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6"
Sep 11 12:06:10.464047 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:10.463857    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f757480-6967-4af8-8517-9110e0956da4-istio-data\") pod \"e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6\" (UID: \"8f757480-6967-4af8-8517-9110e0956da4\") " pod="openshift-ingress/e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6"
Sep 11 12:06:10.464047 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:10.463933    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bq86\" (UniqueName: \"kubernetes.io/projected/8f757480-6967-4af8-8517-9110e0956da4-kube-api-access-9bq86\") pod \"e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6\" (UID: \"8f757480-6967-4af8-8517-9110e0956da4\") " pod="openshift-ingress/e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6"
Sep 11 12:06:10.464047 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:10.463973    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f757480-6967-4af8-8517-9110e0956da4-workload-socket\") pod \"e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6\" (UID: \"8f757480-6967-4af8-8517-9110e0956da4\") " pod="openshift-ingress/e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6"
Sep 11 12:06:10.464047 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:10.464000    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f757480-6967-4af8-8517-9110e0956da4-workload-certs\") pod \"e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6\" (UID: \"8f757480-6967-4af8-8517-9110e0956da4\") " pod="openshift-ingress/e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6"
Sep 11 12:06:10.464047 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:10.464034    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f757480-6967-4af8-8517-9110e0956da4-credential-socket\") pod \"e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6\" (UID: \"8f757480-6967-4af8-8517-9110e0956da4\") " pod="openshift-ingress/e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6"
Sep 11 12:06:10.464239 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:10.464054    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f757480-6967-4af8-8517-9110e0956da4-istio-podinfo\") pod \"e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6\" (UID: \"8f757480-6967-4af8-8517-9110e0956da4\") " pod="openshift-ingress/e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6"
Sep 11 12:06:10.464239 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:10.464091    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f757480-6967-4af8-8517-9110e0956da4-istio-token\") pod \"e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6\" (UID: \"8f757480-6967-4af8-8517-9110e0956da4\") " pod="openshift-ingress/e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6"
Sep 11 12:06:10.564696 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:10.564657    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f757480-6967-4af8-8517-9110e0956da4-credential-socket\") pod \"e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6\" (UID: \"8f757480-6967-4af8-8517-9110e0956da4\") " pod="openshift-ingress/e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6"
Sep 11 12:06:10.564696 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:10.564697    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f757480-6967-4af8-8517-9110e0956da4-istio-podinfo\") pod \"e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6\" (UID: \"8f757480-6967-4af8-8517-9110e0956da4\") " pod="openshift-ingress/e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6"
Sep 11 12:06:10.564977 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:10.564738    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f757480-6967-4af8-8517-9110e0956da4-istio-token\") pod \"e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6\" (UID: \"8f757480-6967-4af8-8517-9110e0956da4\") " pod="openshift-ingress/e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6"
Sep 11 12:06:10.564977 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:10.564797    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f757480-6967-4af8-8517-9110e0956da4-istio-envoy\") pod \"e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6\" (UID: \"8f757480-6967-4af8-8517-9110e0956da4\") " pod="openshift-ingress/e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6"
Sep 11 12:06:10.564977 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:10.564833    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f757480-6967-4af8-8517-9110e0956da4-istiod-ca-cert\") pod \"e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6\" (UID: \"8f757480-6967-4af8-8517-9110e0956da4\") " pod="openshift-ingress/e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6"
Sep 11 12:06:10.564977 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:10.564878    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f757480-6967-4af8-8517-9110e0956da4-istio-data\") pod \"e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6\" (UID: \"8f757480-6967-4af8-8517-9110e0956da4\") " pod="openshift-ingress/e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6"
Sep 11 12:06:10.564977 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:10.564952    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bq86\" (UniqueName: \"kubernetes.io/projected/8f757480-6967-4af8-8517-9110e0956da4-kube-api-access-9bq86\") pod \"e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6\" (UID: \"8f757480-6967-4af8-8517-9110e0956da4\") " pod="openshift-ingress/e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6"
Sep 11 12:06:10.565264 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:10.564984    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f757480-6967-4af8-8517-9110e0956da4-workload-socket\") pod \"e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6\" (UID: \"8f757480-6967-4af8-8517-9110e0956da4\") " pod="openshift-ingress/e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6"
Sep 11 12:06:10.565264 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:10.565018    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f757480-6967-4af8-8517-9110e0956da4-workload-certs\") pod \"e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6\" (UID: \"8f757480-6967-4af8-8517-9110e0956da4\") " pod="openshift-ingress/e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6"
Sep 11 12:06:10.565264 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:10.565178    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f757480-6967-4af8-8517-9110e0956da4-credential-socket\") pod \"e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6\" (UID: \"8f757480-6967-4af8-8517-9110e0956da4\") " pod="openshift-ingress/e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6"
Sep 11 12:06:10.565500 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:10.565330    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f757480-6967-4af8-8517-9110e0956da4-workload-certs\") pod \"e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6\" (UID: \"8f757480-6967-4af8-8517-9110e0956da4\") " pod="openshift-ingress/e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6"
Sep 11 12:06:10.565562 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:10.565520    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f757480-6967-4af8-8517-9110e0956da4-workload-socket\") pod \"e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6\" (UID: \"8f757480-6967-4af8-8517-9110e0956da4\") " pod="openshift-ingress/e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6"
Sep 11 12:06:10.565623 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:10.565609    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f757480-6967-4af8-8517-9110e0956da4-istio-data\") pod \"e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6\" (UID: \"8f757480-6967-4af8-8517-9110e0956da4\") " pod="openshift-ingress/e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6"
Sep 11 12:06:10.565675 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:10.565613    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f757480-6967-4af8-8517-9110e0956da4-istiod-ca-cert\") pod \"e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6\" (UID: \"8f757480-6967-4af8-8517-9110e0956da4\") " pod="openshift-ingress/e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6"
Sep 11 12:06:10.567278 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:10.567256    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f757480-6967-4af8-8517-9110e0956da4-istio-envoy\") pod \"e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6\" (UID: \"8f757480-6967-4af8-8517-9110e0956da4\") " pod="openshift-ingress/e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6"
Sep 11 12:06:10.567433 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:10.567404    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f757480-6967-4af8-8517-9110e0956da4-istio-podinfo\") pod \"e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6\" (UID: \"8f757480-6967-4af8-8517-9110e0956da4\") " pod="openshift-ingress/e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6"
Sep 11 12:06:10.575414 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:10.575342    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f757480-6967-4af8-8517-9110e0956da4-istio-token\") pod \"e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6\" (UID: \"8f757480-6967-4af8-8517-9110e0956da4\") " pod="openshift-ingress/e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6"
Sep 11 12:06:10.576031 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:10.576010    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bq86\" (UniqueName: \"kubernetes.io/projected/8f757480-6967-4af8-8517-9110e0956da4-kube-api-access-9bq86\") pod \"e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6\" (UID: \"8f757480-6967-4af8-8517-9110e0956da4\") " pod="openshift-ingress/e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6"
Sep 11 12:06:10.607771 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:10.607734    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:10.653168 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:10.653135    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6"
Sep 11 12:06:10.768388 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:10.768349    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:06:10.933605 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:10.933581    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6"]
Sep 11 12:06:10.935641 ip-10-0-142-145 kubenswrapper[2579]: W0911 12:06:10.935608    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f757480_6967_4af8_8517_9110e0956da4.slice/crio-80499b994355cb9adcea8c80d5f5658521e665ec2adbcde856e42a53b7ae4c0d WatchSource:0}: Error finding container 80499b994355cb9adcea8c80d5f5658521e665ec2adbcde856e42a53b7ae4c0d: Status 404 returned error can't find the container with id 80499b994355cb9adcea8c80d5f5658521e665ec2adbcde856e42a53b7ae4c0d
Sep 11 12:06:10.937861 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:10.937824    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 12:06:10.937959 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:10.937888    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 12:06:10.937959 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:10.937913    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 12:06:11.647410 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:11.647334    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:11.775845 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:11.775814    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:11.858521 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:11.858485    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:11.866188 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:11.866158    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6" event={"ID":"8f757480-6967-4af8-8517-9110e0956da4","Type":"ContainerStarted","Data":"9b0e46d14805685e3ba03681bc47fa6cc91d6a671769b7f140655470ca661fc7"}
Sep 11 12:06:11.866325 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:11.866211    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6" event={"ID":"8f757480-6967-4af8-8517-9110e0956da4","Type":"ContainerStarted","Data":"80499b994355cb9adcea8c80d5f5658521e665ec2adbcde856e42a53b7ae4c0d"}
Sep 11 12:06:11.892730 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:11.892117    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6" podStartSLOduration=1.892085765 podStartE2EDuration="1.892085765s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:06:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:06:11.889106411 +0000 UTC m=+1075.786548197" watchObservedRunningTime="2026-09-11 12:06:11.892085765 +0000 UTC m=+1075.789527513"
Sep 11 12:06:12.223738 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:12.223696    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:12.316951 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:12.316904    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:12.418754 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:12.418721    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:12.653962 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:12.653926    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6"
Sep 11 12:06:12.659184 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:12.659157    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6"
Sep 11 12:06:12.872541 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:12.872506    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6"
Sep 11 12:06:12.873473 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:12.873455    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6"
Sep 11 12:06:20.383906 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:20.383874    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:22.357614 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:22.357581    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:23.108095 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:23.108051    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:29.613607 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:29.613573    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:29.732510 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:29.732458    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:29.835453 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:29.835422    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:29.937873 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:29.937779    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:30.524038 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:30.523990    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:30.620847 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:30.620811    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:30.691231 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:30.691195    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:32.075490 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:32.075448    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:33.202708 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:33.202673    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:33.307139 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:33.307104    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6"]
Sep 11 12:06:33.307375 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:33.307352    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6" podUID="8f757480-6967-4af8-8517-9110e0956da4" containerName="istio-proxy" containerID="cri-o://9b0e46d14805685e3ba03681bc47fa6cc91d6a671769b7f140655470ca661fc7" gracePeriod=30
Sep 11 12:06:33.375947 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:33.375899    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:33.491878 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:33.491777    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:33.644800 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:33.644761    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:33.760008 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:33.759927    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:06:38.225729 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:38.225688    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:38.565436 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:38.565403    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6"
Sep 11 12:06:38.569871 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:38.569823    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:38.638639 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:38.638596    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f757480-6967-4af8-8517-9110e0956da4-workload-certs\") pod \"8f757480-6967-4af8-8517-9110e0956da4\" (UID: \"8f757480-6967-4af8-8517-9110e0956da4\") "
Sep 11 12:06:38.638639 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:38.638646    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9bq86\" (UniqueName: \"kubernetes.io/projected/8f757480-6967-4af8-8517-9110e0956da4-kube-api-access-9bq86\") pod \"8f757480-6967-4af8-8517-9110e0956da4\" (UID: \"8f757480-6967-4af8-8517-9110e0956da4\") "
Sep 11 12:06:38.638873 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:38.638666    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f757480-6967-4af8-8517-9110e0956da4-istio-envoy\") pod \"8f757480-6967-4af8-8517-9110e0956da4\" (UID: \"8f757480-6967-4af8-8517-9110e0956da4\") "
Sep 11 12:06:38.638873 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:38.638697    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f757480-6967-4af8-8517-9110e0956da4-istio-data\") pod \"8f757480-6967-4af8-8517-9110e0956da4\" (UID: \"8f757480-6967-4af8-8517-9110e0956da4\") "
Sep 11 12:06:38.638873 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:38.638714    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f757480-6967-4af8-8517-9110e0956da4-istiod-ca-cert\") pod \"8f757480-6967-4af8-8517-9110e0956da4\" (UID: \"8f757480-6967-4af8-8517-9110e0956da4\") "
Sep 11 12:06:38.638873 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:38.638736    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f757480-6967-4af8-8517-9110e0956da4-istio-podinfo\") pod \"8f757480-6967-4af8-8517-9110e0956da4\" (UID: \"8f757480-6967-4af8-8517-9110e0956da4\") "
Sep 11 12:06:38.638873 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:38.638788    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f757480-6967-4af8-8517-9110e0956da4-credential-socket\") pod \"8f757480-6967-4af8-8517-9110e0956da4\" (UID: \"8f757480-6967-4af8-8517-9110e0956da4\") "
Sep 11 12:06:38.638873 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:38.638873    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f757480-6967-4af8-8517-9110e0956da4-istio-token\") pod \"8f757480-6967-4af8-8517-9110e0956da4\" (UID: \"8f757480-6967-4af8-8517-9110e0956da4\") "
Sep 11 12:06:38.639182 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:38.638898    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f757480-6967-4af8-8517-9110e0956da4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8f757480-6967-4af8-8517-9110e0956da4" (UID: "8f757480-6967-4af8-8517-9110e0956da4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:06:38.639182 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:38.638916    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f757480-6967-4af8-8517-9110e0956da4-workload-socket\") pod \"8f757480-6967-4af8-8517-9110e0956da4\" (UID: \"8f757480-6967-4af8-8517-9110e0956da4\") "
Sep 11 12:06:38.639182 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:38.639078    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f757480-6967-4af8-8517-9110e0956da4-istio-data" (OuterVolumeSpecName: "istio-data") pod "8f757480-6967-4af8-8517-9110e0956da4" (UID: "8f757480-6967-4af8-8517-9110e0956da4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:06:38.639182 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:38.639153    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f757480-6967-4af8-8517-9110e0956da4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8f757480-6967-4af8-8517-9110e0956da4" (UID: "8f757480-6967-4af8-8517-9110e0956da4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:06:38.639182 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:38.639161    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f757480-6967-4af8-8517-9110e0956da4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8f757480-6967-4af8-8517-9110e0956da4" (UID: "8f757480-6967-4af8-8517-9110e0956da4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:06:38.639446 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:38.639429    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f757480-6967-4af8-8517-9110e0956da4-istio-data\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:06:38.639502 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:38.639454    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f757480-6967-4af8-8517-9110e0956da4-credential-socket\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:06:38.639502 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:38.639471    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f757480-6967-4af8-8517-9110e0956da4-workload-socket\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:06:38.639502 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:38.639484    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f757480-6967-4af8-8517-9110e0956da4-workload-certs\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:06:38.639626 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:38.639501    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f757480-6967-4af8-8517-9110e0956da4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8f757480-6967-4af8-8517-9110e0956da4" (UID: "8f757480-6967-4af8-8517-9110e0956da4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:06:38.641144 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:38.641116    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f757480-6967-4af8-8517-9110e0956da4-istio-token" (OuterVolumeSpecName: "istio-token") pod "8f757480-6967-4af8-8517-9110e0956da4" (UID: "8f757480-6967-4af8-8517-9110e0956da4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:06:38.641275 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:38.641141    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f757480-6967-4af8-8517-9110e0956da4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8f757480-6967-4af8-8517-9110e0956da4" (UID: "8f757480-6967-4af8-8517-9110e0956da4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:06:38.641384 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:38.641363    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f757480-6967-4af8-8517-9110e0956da4-kube-api-access-9bq86" (OuterVolumeSpecName: "kube-api-access-9bq86") pod "8f757480-6967-4af8-8517-9110e0956da4" (UID: "8f757480-6967-4af8-8517-9110e0956da4"). InnerVolumeSpecName "kube-api-access-9bq86". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:06:38.641384 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:38.641371    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8f757480-6967-4af8-8517-9110e0956da4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8f757480-6967-4af8-8517-9110e0956da4" (UID: "8f757480-6967-4af8-8517-9110e0956da4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 12:06:38.669639 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:38.669584    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:38.740345 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:38.740258    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f757480-6967-4af8-8517-9110e0956da4-istio-token\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:06:38.740345 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:38.740290    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9bq86\" (UniqueName: \"kubernetes.io/projected/8f757480-6967-4af8-8517-9110e0956da4-kube-api-access-9bq86\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:06:38.740345 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:38.740301    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f757480-6967-4af8-8517-9110e0956da4-istio-envoy\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:06:38.740345 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:38.740312    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f757480-6967-4af8-8517-9110e0956da4-istiod-ca-cert\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:06:38.740345 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:38.740320    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f757480-6967-4af8-8517-9110e0956da4-istio-podinfo\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:06:38.814111 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:38.814079    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:38.917628 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:38.917592    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:39.010689 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:39.010594    2579 generic.go:356] "Generic (PLEG): container finished" podID="8f757480-6967-4af8-8517-9110e0956da4" containerID="9b0e46d14805685e3ba03681bc47fa6cc91d6a671769b7f140655470ca661fc7" exitCode=0
Sep 11 12:06:39.010689 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:39.010642    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6"
Sep 11 12:06:39.010689 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:39.010675    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6" event={"ID":"8f757480-6967-4af8-8517-9110e0956da4","Type":"ContainerDied","Data":"9b0e46d14805685e3ba03681bc47fa6cc91d6a671769b7f140655470ca661fc7"}
Sep 11 12:06:39.010941 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:39.010727    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6" event={"ID":"8f757480-6967-4af8-8517-9110e0956da4","Type":"ContainerDied","Data":"80499b994355cb9adcea8c80d5f5658521e665ec2adbcde856e42a53b7ae4c0d"}
Sep 11 12:06:39.010941 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:39.010749    2579 scope.go:117] "RemoveContainer" containerID="9b0e46d14805685e3ba03681bc47fa6cc91d6a671769b7f140655470ca661fc7"
Sep 11 12:06:39.020981 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:39.020960    2579 scope.go:117] "RemoveContainer" containerID="9b0e46d14805685e3ba03681bc47fa6cc91d6a671769b7f140655470ca661fc7"
Sep 11 12:06:39.021329 ip-10-0-142-145 kubenswrapper[2579]: E0911 12:06:39.021304    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b0e46d14805685e3ba03681bc47fa6cc91d6a671769b7f140655470ca661fc7\": container with ID starting with 9b0e46d14805685e3ba03681bc47fa6cc91d6a671769b7f140655470ca661fc7 not found: ID does not exist" containerID="9b0e46d14805685e3ba03681bc47fa6cc91d6a671769b7f140655470ca661fc7"
Sep 11 12:06:39.021548 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:39.021336    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b0e46d14805685e3ba03681bc47fa6cc91d6a671769b7f140655470ca661fc7"} err="failed to get container status \"9b0e46d14805685e3ba03681bc47fa6cc91d6a671769b7f140655470ca661fc7\": rpc error: code = NotFound desc = could not find container \"9b0e46d14805685e3ba03681bc47fa6cc91d6a671769b7f140655470ca661fc7\": container with ID starting with 9b0e46d14805685e3ba03681bc47fa6cc91d6a671769b7f140655470ca661fc7 not found: ID does not exist"
Sep 11 12:06:39.042256 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:39.042224    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6"]
Sep 11 12:06:39.048609 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:39.048578    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-noten-4a674b-openshift-default-5dcc666664-w7bq6"]
Sep 11 12:06:39.269789 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:39.269697    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:39.515032 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:39.515002    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:39.615116 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:39.615080    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:40.576816 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:40.576780    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f757480-6967-4af8-8517-9110e0956da4" path="/var/lib/kubelet/pods/8f757480-6967-4af8-8517-9110e0956da4/volumes"
Sep 11 12:06:45.561902 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:45.561870    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:45.637543 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:45.637511    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:46.163735 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:46.163704    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9"]
Sep 11 12:06:46.164375 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:46.164351    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f757480-6967-4af8-8517-9110e0956da4" containerName="istio-proxy"
Sep 11 12:06:46.164375 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:46.164372    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f757480-6967-4af8-8517-9110e0956da4" containerName="istio-proxy"
Sep 11 12:06:46.164545 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:46.164470    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f757480-6967-4af8-8517-9110e0956da4" containerName="istio-proxy"
Sep 11 12:06:46.167362 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:46.167344    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9"
Sep 11 12:06:46.171685 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:46.171664    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-24cfea9d-openshift-default-dockercfg-lsdpn\""
Sep 11 12:06:46.183309 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:46.183282    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9"]
Sep 11 12:06:46.312254 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:46.312208    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4mz5\" (UniqueName: \"kubernetes.io/projected/16aaa4c7-eb22-436e-ab60-12d4c78a732e-kube-api-access-s4mz5\") pod \"e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9\" (UID: \"16aaa4c7-eb22-436e-ab60-12d4c78a732e\") " pod="openshift-ingress/e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9"
Sep 11 12:06:46.312482 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:46.312267    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/16aaa4c7-eb22-436e-ab60-12d4c78a732e-istio-token\") pod \"e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9\" (UID: \"16aaa4c7-eb22-436e-ab60-12d4c78a732e\") " pod="openshift-ingress/e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9"
Sep 11 12:06:46.312482 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:46.312294    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/16aaa4c7-eb22-436e-ab60-12d4c78a732e-workload-certs\") pod \"e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9\" (UID: \"16aaa4c7-eb22-436e-ab60-12d4c78a732e\") " pod="openshift-ingress/e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9"
Sep 11 12:06:46.312482 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:46.312324    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/16aaa4c7-eb22-436e-ab60-12d4c78a732e-istiod-ca-cert\") pod \"e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9\" (UID: \"16aaa4c7-eb22-436e-ab60-12d4c78a732e\") " pod="openshift-ingress/e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9"
Sep 11 12:06:46.312482 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:46.312356    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/16aaa4c7-eb22-436e-ab60-12d4c78a732e-workload-socket\") pod \"e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9\" (UID: \"16aaa4c7-eb22-436e-ab60-12d4c78a732e\") " pod="openshift-ingress/e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9"
Sep 11 12:06:46.312482 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:46.312431    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/16aaa4c7-eb22-436e-ab60-12d4c78a732e-istio-podinfo\") pod \"e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9\" (UID: \"16aaa4c7-eb22-436e-ab60-12d4c78a732e\") " pod="openshift-ingress/e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9"
Sep 11 12:06:46.312685 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:46.312501    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/16aaa4c7-eb22-436e-ab60-12d4c78a732e-credential-socket\") pod \"e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9\" (UID: \"16aaa4c7-eb22-436e-ab60-12d4c78a732e\") " pod="openshift-ingress/e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9"
Sep 11 12:06:46.312685 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:46.312533    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/16aaa4c7-eb22-436e-ab60-12d4c78a732e-istio-envoy\") pod \"e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9\" (UID: \"16aaa4c7-eb22-436e-ab60-12d4c78a732e\") " pod="openshift-ingress/e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9"
Sep 11 12:06:46.312685 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:46.312552    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/16aaa4c7-eb22-436e-ab60-12d4c78a732e-istio-data\") pod \"e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9\" (UID: \"16aaa4c7-eb22-436e-ab60-12d4c78a732e\") " pod="openshift-ingress/e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9"
Sep 11 12:06:46.413287 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:46.413244    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/16aaa4c7-eb22-436e-ab60-12d4c78a732e-credential-socket\") pod \"e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9\" (UID: \"16aaa4c7-eb22-436e-ab60-12d4c78a732e\") " pod="openshift-ingress/e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9"
Sep 11 12:06:46.413484 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:46.413303    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/16aaa4c7-eb22-436e-ab60-12d4c78a732e-istio-envoy\") pod \"e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9\" (UID: \"16aaa4c7-eb22-436e-ab60-12d4c78a732e\") " pod="openshift-ingress/e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9"
Sep 11 12:06:46.413484 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:46.413337    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/16aaa4c7-eb22-436e-ab60-12d4c78a732e-istio-data\") pod \"e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9\" (UID: \"16aaa4c7-eb22-436e-ab60-12d4c78a732e\") " pod="openshift-ingress/e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9"
Sep 11 12:06:46.413484 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:46.413370    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4mz5\" (UniqueName: \"kubernetes.io/projected/16aaa4c7-eb22-436e-ab60-12d4c78a732e-kube-api-access-s4mz5\") pod \"e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9\" (UID: \"16aaa4c7-eb22-436e-ab60-12d4c78a732e\") " pod="openshift-ingress/e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9"
Sep 11 12:06:46.413484 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:46.413450    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/16aaa4c7-eb22-436e-ab60-12d4c78a732e-istio-token\") pod \"e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9\" (UID: \"16aaa4c7-eb22-436e-ab60-12d4c78a732e\") " pod="openshift-ingress/e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9"
Sep 11 12:06:46.413720 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:46.413490    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/16aaa4c7-eb22-436e-ab60-12d4c78a732e-workload-certs\") pod \"e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9\" (UID: \"16aaa4c7-eb22-436e-ab60-12d4c78a732e\") " pod="openshift-ingress/e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9"
Sep 11 12:06:46.413720 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:46.413628    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/16aaa4c7-eb22-436e-ab60-12d4c78a732e-istiod-ca-cert\") pod \"e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9\" (UID: \"16aaa4c7-eb22-436e-ab60-12d4c78a732e\") " pod="openshift-ingress/e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9"
Sep 11 12:06:46.413720 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:46.413681    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/16aaa4c7-eb22-436e-ab60-12d4c78a732e-credential-socket\") pod \"e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9\" (UID: \"16aaa4c7-eb22-436e-ab60-12d4c78a732e\") " pod="openshift-ingress/e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9"
Sep 11 12:06:46.413720 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:46.413713    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/16aaa4c7-eb22-436e-ab60-12d4c78a732e-workload-socket\") pod \"e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9\" (UID: \"16aaa4c7-eb22-436e-ab60-12d4c78a732e\") " pod="openshift-ingress/e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9"
Sep 11 12:06:46.413948 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:46.413755    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/16aaa4c7-eb22-436e-ab60-12d4c78a732e-istio-data\") pod \"e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9\" (UID: \"16aaa4c7-eb22-436e-ab60-12d4c78a732e\") " pod="openshift-ingress/e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9"
Sep 11 12:06:46.413948 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:46.413775    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/16aaa4c7-eb22-436e-ab60-12d4c78a732e-istio-podinfo\") pod \"e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9\" (UID: \"16aaa4c7-eb22-436e-ab60-12d4c78a732e\") " pod="openshift-ingress/e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9"
Sep 11 12:06:46.414041 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:46.414012    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/16aaa4c7-eb22-436e-ab60-12d4c78a732e-workload-socket\") pod \"e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9\" (UID: \"16aaa4c7-eb22-436e-ab60-12d4c78a732e\") " pod="openshift-ingress/e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9"
Sep 11 12:06:46.414230 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:46.414203    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/16aaa4c7-eb22-436e-ab60-12d4c78a732e-workload-certs\") pod \"e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9\" (UID: \"16aaa4c7-eb22-436e-ab60-12d4c78a732e\") " pod="openshift-ingress/e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9"
Sep 11 12:06:46.414374 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:46.414289    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/16aaa4c7-eb22-436e-ab60-12d4c78a732e-istiod-ca-cert\") pod \"e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9\" (UID: \"16aaa4c7-eb22-436e-ab60-12d4c78a732e\") " pod="openshift-ingress/e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9"
Sep 11 12:06:46.415719 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:46.415702    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/16aaa4c7-eb22-436e-ab60-12d4c78a732e-istio-envoy\") pod \"e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9\" (UID: \"16aaa4c7-eb22-436e-ab60-12d4c78a732e\") " pod="openshift-ingress/e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9"
Sep 11 12:06:46.415958 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:46.415941    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/16aaa4c7-eb22-436e-ab60-12d4c78a732e-istio-podinfo\") pod \"e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9\" (UID: \"16aaa4c7-eb22-436e-ab60-12d4c78a732e\") " pod="openshift-ingress/e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9"
Sep 11 12:06:46.422355 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:46.422325    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/16aaa4c7-eb22-436e-ab60-12d4c78a732e-istio-token\") pod \"e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9\" (UID: \"16aaa4c7-eb22-436e-ab60-12d4c78a732e\") " pod="openshift-ingress/e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9"
Sep 11 12:06:46.422501 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:46.422484    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4mz5\" (UniqueName: \"kubernetes.io/projected/16aaa4c7-eb22-436e-ab60-12d4c78a732e-kube-api-access-s4mz5\") pod \"e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9\" (UID: \"16aaa4c7-eb22-436e-ab60-12d4c78a732e\") " pod="openshift-ingress/e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9"
Sep 11 12:06:46.478248 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:46.478207    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9"
Sep 11 12:06:46.756223 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:46.756196    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9"]
Sep 11 12:06:46.758067 ip-10-0-142-145 kubenswrapper[2579]: W0911 12:06:46.758034    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16aaa4c7_eb22_436e_ab60_12d4c78a732e.slice/crio-bc416b4a63294a6b165986930bf95fa845c4916b456cf869e0fe21500a811c40 WatchSource:0}: Error finding container bc416b4a63294a6b165986930bf95fa845c4916b456cf869e0fe21500a811c40: Status 404 returned error can't find the container with id bc416b4a63294a6b165986930bf95fa845c4916b456cf869e0fe21500a811c40
Sep 11 12:06:46.760357 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:46.760312    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 12:06:46.760475 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:46.760437    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 12:06:46.760517 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:46.760492    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 12:06:46.772553 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:46.772523    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:06:47.054840 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:47.054753    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9" event={"ID":"16aaa4c7-eb22-436e-ab60-12d4c78a732e","Type":"ContainerStarted","Data":"c65b4c5b32b567b61eac3d928d7db6fd2253a21cbec6c0cbc960f21f5105f219"}
Sep 11 12:06:47.054840 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:47.054817    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9" event={"ID":"16aaa4c7-eb22-436e-ab60-12d4c78a732e","Type":"ContainerStarted","Data":"bc416b4a63294a6b165986930bf95fa845c4916b456cf869e0fe21500a811c40"}
Sep 11 12:06:47.081020 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:47.080961    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9" podStartSLOduration=1.080944959 podStartE2EDuration="1.080944959s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:06:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:06:47.078425257 +0000 UTC m=+1110.975867002" watchObservedRunningTime="2026-09-11 12:06:47.080944959 +0000 UTC m=+1110.978386705"
Sep 11 12:06:47.459771 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:47.459742    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:47.478526 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:47.478496    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9"
Sep 11 12:06:47.484120 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:47.484091    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9"
Sep 11 12:06:47.608633 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:47.608596    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:48.059869 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:48.059839    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9"
Sep 11 12:06:48.060835 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:48.060816    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9"
Sep 11 12:06:49.067998 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:49.067965    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:49.185694 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:49.185661    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:50.096861 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:50.096822    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:50.126203 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:50.126169    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-cbc380-d9ccbfcb7-zrzjq"]
Sep 11 12:06:50.126480 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:50.126447    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-cbc380-d9ccbfcb7-zrzjq" podUID="2f7a23ed-b195-4dcb-a35a-466ebde97aae" containerName="payload-processing" containerID="cri-o://0741a9251bc6c03ddaebb186f04958c24d2fa6ec509d0d3980e6163b212e98cd" gracePeriod=30
Sep 11 12:06:50.196138 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:50.196110    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:50.297125 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:50.297098    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:50.386269 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:50.386241    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-cbc380-d9ccbfcb7-zrzjq"
Sep 11 12:06:50.398247 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:50.398214    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:50.474890 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:50.474858    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:50.555896 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:50.555864    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f7a23ed-b195-4dcb-a35a-466ebde97aae-plugins-config-volume\") pod \"2f7a23ed-b195-4dcb-a35a-466ebde97aae\" (UID: \"2f7a23ed-b195-4dcb-a35a-466ebde97aae\") "
Sep 11 12:06:50.556074 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:50.555958    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wd4sr\" (UniqueName: \"kubernetes.io/projected/2f7a23ed-b195-4dcb-a35a-466ebde97aae-kube-api-access-wd4sr\") pod \"2f7a23ed-b195-4dcb-a35a-466ebde97aae\" (UID: \"2f7a23ed-b195-4dcb-a35a-466ebde97aae\") "
Sep 11 12:06:50.556313 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:50.556286    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f7a23ed-b195-4dcb-a35a-466ebde97aae-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2f7a23ed-b195-4dcb-a35a-466ebde97aae" (UID: "2f7a23ed-b195-4dcb-a35a-466ebde97aae"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:06:50.557989 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:50.557963    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f7a23ed-b195-4dcb-a35a-466ebde97aae-kube-api-access-wd4sr" (OuterVolumeSpecName: "kube-api-access-wd4sr") pod "2f7a23ed-b195-4dcb-a35a-466ebde97aae" (UID: "2f7a23ed-b195-4dcb-a35a-466ebde97aae"). InnerVolumeSpecName "kube-api-access-wd4sr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:06:50.657758 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:50.657660    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wd4sr\" (UniqueName: \"kubernetes.io/projected/2f7a23ed-b195-4dcb-a35a-466ebde97aae-kube-api-access-wd4sr\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:06:50.657758 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:50.657697    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f7a23ed-b195-4dcb-a35a-466ebde97aae-plugins-config-volume\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:06:50.786267 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:50.786229    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:51.077680 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:51.077582    2579 generic.go:356] "Generic (PLEG): container finished" podID="2f7a23ed-b195-4dcb-a35a-466ebde97aae" containerID="0741a9251bc6c03ddaebb186f04958c24d2fa6ec509d0d3980e6163b212e98cd" exitCode=0
Sep 11 12:06:51.077680 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:51.077658    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-cbc380-d9ccbfcb7-zrzjq"
Sep 11 12:06:51.077680 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:51.077664    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-cbc380-d9ccbfcb7-zrzjq" event={"ID":"2f7a23ed-b195-4dcb-a35a-466ebde97aae","Type":"ContainerDied","Data":"0741a9251bc6c03ddaebb186f04958c24d2fa6ec509d0d3980e6163b212e98cd"}
Sep 11 12:06:51.077953 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:51.077768    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-cbc380-d9ccbfcb7-zrzjq" event={"ID":"2f7a23ed-b195-4dcb-a35a-466ebde97aae","Type":"ContainerDied","Data":"83e28c0a2d0ce0494359b5687b77ae5ed9a47fa27e5837dbc052e596cec2ed63"}
Sep 11 12:06:51.077953 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:51.077802    2579 scope.go:117] "RemoveContainer" containerID="0741a9251bc6c03ddaebb186f04958c24d2fa6ec509d0d3980e6163b212e98cd"
Sep 11 12:06:51.089343 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:51.089293    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:51.090481 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:51.090455    2579 scope.go:117] "RemoveContainer" containerID="0741a9251bc6c03ddaebb186f04958c24d2fa6ec509d0d3980e6163b212e98cd"
Sep 11 12:06:51.090880 ip-10-0-142-145 kubenswrapper[2579]: E0911 12:06:51.090859    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0741a9251bc6c03ddaebb186f04958c24d2fa6ec509d0d3980e6163b212e98cd\": container with ID starting with 0741a9251bc6c03ddaebb186f04958c24d2fa6ec509d0d3980e6163b212e98cd not found: ID does not exist" containerID="0741a9251bc6c03ddaebb186f04958c24d2fa6ec509d0d3980e6163b212e98cd"
Sep 11 12:06:51.090973 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:51.090890    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0741a9251bc6c03ddaebb186f04958c24d2fa6ec509d0d3980e6163b212e98cd"} err="failed to get container status \"0741a9251bc6c03ddaebb186f04958c24d2fa6ec509d0d3980e6163b212e98cd\": rpc error: code = NotFound desc = could not find container \"0741a9251bc6c03ddaebb186f04958c24d2fa6ec509d0d3980e6163b212e98cd\": container with ID starting with 0741a9251bc6c03ddaebb186f04958c24d2fa6ec509d0d3980e6163b212e98cd not found: ID does not exist"
Sep 11 12:06:51.115039 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:51.115000    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-cbc380-d9ccbfcb7-zrzjq"]
Sep 11 12:06:51.124385 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:51.124353    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-cbc380-d9ccbfcb7-zrzjq"]
Sep 11 12:06:51.178766 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:51.178728    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:51.256197 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:51.256163    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-24cfea9d-7b69fc9c8b-bh8ds"]
Sep 11 12:06:51.256728 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:51.256704    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f7a23ed-b195-4dcb-a35a-466ebde97aae" containerName="payload-processing"
Sep 11 12:06:51.256728 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:51.256728    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f7a23ed-b195-4dcb-a35a-466ebde97aae" containerName="payload-processing"
Sep 11 12:06:51.256908 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:51.256872    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f7a23ed-b195-4dcb-a35a-466ebde97aae" containerName="payload-processing"
Sep 11 12:06:51.260023 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:51.260002    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-24cfea9d-7b69fc9c8b-bh8ds"
Sep 11 12:06:51.262633 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:51.262610    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-24cfea9d\""
Sep 11 12:06:51.262742 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:51.262628    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-24cfea9d-dockercfg-vwgnm\""
Sep 11 12:06:51.271870 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:51.271841    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-24cfea9d-7b69fc9c8b-bh8ds"]
Sep 11 12:06:51.290319 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:51.290281    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:51.364554 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:51.364514    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80ab63cd-18b4-42fd-ae10-22f7dac14c49-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-24cfea9d-7b69fc9c8b-bh8ds\" (UID: \"80ab63cd-18b4-42fd-ae10-22f7dac14c49\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-24cfea9d-7b69fc9c8b-bh8ds"
Sep 11 12:06:51.364771 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:51.364575    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fss5n\" (UniqueName: \"kubernetes.io/projected/80ab63cd-18b4-42fd-ae10-22f7dac14c49-kube-api-access-fss5n\") pod \"payload-pre-processing-e2e-mt-24cfea9d-7b69fc9c8b-bh8ds\" (UID: \"80ab63cd-18b4-42fd-ae10-22f7dac14c49\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-24cfea9d-7b69fc9c8b-bh8ds"
Sep 11 12:06:51.465370 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:51.465311    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80ab63cd-18b4-42fd-ae10-22f7dac14c49-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-24cfea9d-7b69fc9c8b-bh8ds\" (UID: \"80ab63cd-18b4-42fd-ae10-22f7dac14c49\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-24cfea9d-7b69fc9c8b-bh8ds"
Sep 11 12:06:51.465607 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:51.465387    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fss5n\" (UniqueName: \"kubernetes.io/projected/80ab63cd-18b4-42fd-ae10-22f7dac14c49-kube-api-access-fss5n\") pod \"payload-pre-processing-e2e-mt-24cfea9d-7b69fc9c8b-bh8ds\" (UID: \"80ab63cd-18b4-42fd-ae10-22f7dac14c49\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-24cfea9d-7b69fc9c8b-bh8ds"
Sep 11 12:06:51.466135 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:51.466115    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80ab63cd-18b4-42fd-ae10-22f7dac14c49-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-24cfea9d-7b69fc9c8b-bh8ds\" (UID: \"80ab63cd-18b4-42fd-ae10-22f7dac14c49\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-24cfea9d-7b69fc9c8b-bh8ds"
Sep 11 12:06:51.474705 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:51.474672    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fss5n\" (UniqueName: \"kubernetes.io/projected/80ab63cd-18b4-42fd-ae10-22f7dac14c49-kube-api-access-fss5n\") pod \"payload-pre-processing-e2e-mt-24cfea9d-7b69fc9c8b-bh8ds\" (UID: \"80ab63cd-18b4-42fd-ae10-22f7dac14c49\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-24cfea9d-7b69fc9c8b-bh8ds"
Sep 11 12:06:51.570875 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:51.570832    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-24cfea9d-7b69fc9c8b-bh8ds"
Sep 11 12:06:51.846219 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:51.846190    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-24cfea9d-7b69fc9c8b-bh8ds"]
Sep 11 12:06:51.848258 ip-10-0-142-145 kubenswrapper[2579]: W0911 12:06:51.848235    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80ab63cd_18b4_42fd_ae10_22f7dac14c49.slice/crio-ecad6b22eb43af4b447db12a25e7e8f5bb0d63ace295d9a86e22a0fa52f16c4c WatchSource:0}: Error finding container ecad6b22eb43af4b447db12a25e7e8f5bb0d63ace295d9a86e22a0fa52f16c4c: Status 404 returned error can't find the container with id ecad6b22eb43af4b447db12a25e7e8f5bb0d63ace295d9a86e22a0fa52f16c4c
Sep 11 12:06:52.085706 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:52.085671    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-24cfea9d-7b69fc9c8b-bh8ds" event={"ID":"80ab63cd-18b4-42fd-ae10-22f7dac14c49","Type":"ContainerStarted","Data":"e8c3d463c969eaeead544c5b78cf98cf15e37e8d9925411f57f4ab534c59a3e8"}
Sep 11 12:06:52.085902 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:52.085723    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-24cfea9d-7b69fc9c8b-bh8ds" event={"ID":"80ab63cd-18b4-42fd-ae10-22f7dac14c49","Type":"ContainerStarted","Data":"ecad6b22eb43af4b447db12a25e7e8f5bb0d63ace295d9a86e22a0fa52f16c4c"}
Sep 11 12:06:52.085902 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:52.085754    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-24cfea9d-7b69fc9c8b-bh8ds"
Sep 11 12:06:52.106531 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:52.106475    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-24cfea9d-7b69fc9c8b-bh8ds" podStartSLOduration=1.106459667 podStartE2EDuration="1.106459667s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:06:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:06:52.103561908 +0000 UTC m=+1116.001003657" watchObservedRunningTime="2026-09-11 12:06:52.106459667 +0000 UTC m=+1116.003901411"
Sep 11 12:06:52.577747 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:52.577710    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f7a23ed-b195-4dcb-a35a-466ebde97aae" path="/var/lib/kubelet/pods/2f7a23ed-b195-4dcb-a35a-466ebde97aae/volumes"
Sep 11 12:06:54.576947 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:54.576915    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:06:57.040514 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:57.040471    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:57.283100 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:57.283053    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:57.348626 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:57.348588    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:57.779077 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:06:57.778977    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:03.094587 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:03.094552    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-24cfea9d-7b69fc9c8b-bh8ds"
Sep 11 12:07:04.101636 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:04.101594    2579 patch_prober.go:28] interesting pod/e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 11 12:07:04.102011 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:04.101670    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9" podUID="16aaa4c7-eb22-436e-ab60-12d4c78a732e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 11 12:07:05.160476 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:05.160443    2579 generic.go:356] "Generic (PLEG): container finished" podID="16aaa4c7-eb22-436e-ab60-12d4c78a732e" containerID="c65b4c5b32b567b61eac3d928d7db6fd2253a21cbec6c0cbc960f21f5105f219" exitCode=0
Sep 11 12:07:05.160920 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:05.160494    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9" event={"ID":"16aaa4c7-eb22-436e-ab60-12d4c78a732e","Type":"ContainerDied","Data":"c65b4c5b32b567b61eac3d928d7db6fd2253a21cbec6c0cbc960f21f5105f219"}
Sep 11 12:07:05.160920 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:05.160910    2579 scope.go:117] "RemoveContainer" containerID="c65b4c5b32b567b61eac3d928d7db6fd2253a21cbec6c0cbc960f21f5105f219"
Sep 11 12:07:05.161987 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:05.161956    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 12:07:05.162079 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:05.162039    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 12:07:05.162124 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:05.162081    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 12:07:06.167045 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:06.167011    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9" event={"ID":"16aaa4c7-eb22-436e-ab60-12d4c78a732e","Type":"ContainerStarted","Data":"062315248de70e9791b5fbebd229ad8a027420b4b1cf998f20c8d1052fa68929"}
Sep 11 12:07:06.479285 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:06.479190    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9"
Sep 11 12:07:06.479572 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:06.479242    2579 patch_prober.go:28] interesting pod/e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 11 12:07:06.479739 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:06.479717    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9" podUID="16aaa4c7-eb22-436e-ab60-12d4c78a732e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 11 12:07:07.479720 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:07.479677    2579 patch_prober.go:28] interesting pod/e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 11 12:07:07.480237 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:07.479753    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9" podUID="16aaa4c7-eb22-436e-ab60-12d4c78a732e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 11 12:07:08.224639 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:08.224597    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-12d2a9-57b894675b-phbq4"]
Sep 11 12:07:08.224966 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:08.224907    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-12d2a9-57b894675b-phbq4" podUID="2418f3a0-c6e5-4a97-b33d-f5f62fd70741" containerName="maas-api" containerID="cri-o://67a38218a967691f68d60bcbbb97fabdad8e6a366c1b748208745d28c8237d6e" gracePeriod=30
Sep 11 12:07:08.254262 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:08.254232    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:08.353143 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:08.353089    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:08.480385 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:08.479858    2579 patch_prober.go:28] interesting pod/e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 11 12:07:08.480385 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:08.479928    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9" podUID="16aaa4c7-eb22-436e-ab60-12d4c78a732e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 11 12:07:08.519025 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:08.518988    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:08.539991 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:08.539966    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-12d2a9-57b894675b-phbq4"
Sep 11 12:07:08.635182 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:08.635144    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2418f3a0-c6e5-4a97-b33d-f5f62fd70741-openshift-service-ca\") pod \"2418f3a0-c6e5-4a97-b33d-f5f62fd70741\" (UID: \"2418f3a0-c6e5-4a97-b33d-f5f62fd70741\") "
Sep 11 12:07:08.635334 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:08.635248    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bs5z5\" (UniqueName: \"kubernetes.io/projected/2418f3a0-c6e5-4a97-b33d-f5f62fd70741-kube-api-access-bs5z5\") pod \"2418f3a0-c6e5-4a97-b33d-f5f62fd70741\" (UID: \"2418f3a0-c6e5-4a97-b33d-f5f62fd70741\") "
Sep 11 12:07:08.635334 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:08.635286    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2418f3a0-c6e5-4a97-b33d-f5f62fd70741-maas-api-tls\") pod \"2418f3a0-c6e5-4a97-b33d-f5f62fd70741\" (UID: \"2418f3a0-c6e5-4a97-b33d-f5f62fd70741\") "
Sep 11 12:07:08.635334 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:08.635316    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2418f3a0-c6e5-4a97-b33d-f5f62fd70741-metrics-certs\") pod \"2418f3a0-c6e5-4a97-b33d-f5f62fd70741\" (UID: \"2418f3a0-c6e5-4a97-b33d-f5f62fd70741\") "
Sep 11 12:07:08.635602 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:08.635574    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2418f3a0-c6e5-4a97-b33d-f5f62fd70741-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2418f3a0-c6e5-4a97-b33d-f5f62fd70741" (UID: "2418f3a0-c6e5-4a97-b33d-f5f62fd70741"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:07:08.635742 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:08.635723    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2418f3a0-c6e5-4a97-b33d-f5f62fd70741-openshift-service-ca\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:07:08.637547 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:08.637518    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2418f3a0-c6e5-4a97-b33d-f5f62fd70741-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "2418f3a0-c6e5-4a97-b33d-f5f62fd70741" (UID: "2418f3a0-c6e5-4a97-b33d-f5f62fd70741"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:07:08.637658 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:08.637541    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2418f3a0-c6e5-4a97-b33d-f5f62fd70741-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2418f3a0-c6e5-4a97-b33d-f5f62fd70741" (UID: "2418f3a0-c6e5-4a97-b33d-f5f62fd70741"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:07:08.637785 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:08.637765    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2418f3a0-c6e5-4a97-b33d-f5f62fd70741-kube-api-access-bs5z5" (OuterVolumeSpecName: "kube-api-access-bs5z5") pod "2418f3a0-c6e5-4a97-b33d-f5f62fd70741" (UID: "2418f3a0-c6e5-4a97-b33d-f5f62fd70741"). InnerVolumeSpecName "kube-api-access-bs5z5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:07:08.722029 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:08.721989    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:08.737006 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:08.736930    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bs5z5\" (UniqueName: \"kubernetes.io/projected/2418f3a0-c6e5-4a97-b33d-f5f62fd70741-kube-api-access-bs5z5\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:07:08.737006 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:08.736959    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2418f3a0-c6e5-4a97-b33d-f5f62fd70741-maas-api-tls\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:07:08.737006 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:08.736969    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2418f3a0-c6e5-4a97-b33d-f5f62fd70741-metrics-certs\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:07:09.192268 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:09.192220    2579 generic.go:356] "Generic (PLEG): container finished" podID="2418f3a0-c6e5-4a97-b33d-f5f62fd70741" containerID="67a38218a967691f68d60bcbbb97fabdad8e6a366c1b748208745d28c8237d6e" exitCode=0
Sep 11 12:07:09.192504 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:09.192362    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-12d2a9-57b894675b-phbq4" event={"ID":"2418f3a0-c6e5-4a97-b33d-f5f62fd70741","Type":"ContainerDied","Data":"67a38218a967691f68d60bcbbb97fabdad8e6a366c1b748208745d28c8237d6e"}
Sep 11 12:07:09.192504 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:09.192439    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-12d2a9-57b894675b-phbq4" event={"ID":"2418f3a0-c6e5-4a97-b33d-f5f62fd70741","Type":"ContainerDied","Data":"5e10096c44c423cf3557a51f49135a9bf8b72bea4efed3bdb2b1384502882277"}
Sep 11 12:07:09.192504 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:09.192474    2579 scope.go:117] "RemoveContainer" containerID="67a38218a967691f68d60bcbbb97fabdad8e6a366c1b748208745d28c8237d6e"
Sep 11 12:07:09.192706 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:09.192639    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-12d2a9-57b894675b-phbq4"
Sep 11 12:07:09.215415 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:09.215371    2579 scope.go:117] "RemoveContainer" containerID="67a38218a967691f68d60bcbbb97fabdad8e6a366c1b748208745d28c8237d6e"
Sep 11 12:07:09.215834 ip-10-0-142-145 kubenswrapper[2579]: E0911 12:07:09.215814    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67a38218a967691f68d60bcbbb97fabdad8e6a366c1b748208745d28c8237d6e\": container with ID starting with 67a38218a967691f68d60bcbbb97fabdad8e6a366c1b748208745d28c8237d6e not found: ID does not exist" containerID="67a38218a967691f68d60bcbbb97fabdad8e6a366c1b748208745d28c8237d6e"
Sep 11 12:07:09.215918 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:09.215846    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67a38218a967691f68d60bcbbb97fabdad8e6a366c1b748208745d28c8237d6e"} err="failed to get container status \"67a38218a967691f68d60bcbbb97fabdad8e6a366c1b748208745d28c8237d6e\": rpc error: code = NotFound desc = could not find container \"67a38218a967691f68d60bcbbb97fabdad8e6a366c1b748208745d28c8237d6e\": container with ID starting with 67a38218a967691f68d60bcbbb97fabdad8e6a366c1b748208745d28c8237d6e not found: ID does not exist"
Sep 11 12:07:09.253465 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:09.253431    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-12d2a9-57b894675b-phbq4"]
Sep 11 12:07:09.262029 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:09.261987    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-12d2a9-57b894675b-phbq4"]
Sep 11 12:07:09.405765 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:09.405735    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:09.479350 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:09.479253    2579 patch_prober.go:28] interesting pod/e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 11 12:07:09.480055 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:09.479655    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9" podUID="16aaa4c7-eb22-436e-ab60-12d4c78a732e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 11 12:07:10.479650 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:10.479610    2579 patch_prober.go:28] interesting pod/e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 11 12:07:10.480160 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:10.479676    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9" podUID="16aaa4c7-eb22-436e-ab60-12d4c78a732e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 11 12:07:10.587419 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:10.576287    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2418f3a0-c6e5-4a97-b33d-f5f62fd70741" path="/var/lib/kubelet/pods/2418f3a0-c6e5-4a97-b33d-f5f62fd70741/volumes"
Sep 11 12:07:11.166480 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:11.166442    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:07:11.482094 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:11.482014    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9"
Sep 11 12:07:11.482584 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:11.482423    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9"
Sep 11 12:07:11.482957 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:11.482939    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9"
Sep 11 12:07:11.583613 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:11.583581    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:14.239666 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:14.239630    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:14.969892 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:14.969847    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:15.088362 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:15.088329    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:15.369416 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:15.369355    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:16.744204 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:16.744165    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv"]
Sep 11 12:07:16.744604 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:16.744463    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv" podUID="36e69a19-0f39-41aa-b80e-e60b2b9cf3c7" containerName="istio-proxy" containerID="cri-o://c6587bc5c46ed78642d9cf1395f52b68f153af5e25cfeb65a33d1cfe68d081cf" gracePeriod=30
Sep 11 12:07:17.493782 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:17.493745    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:17.597871 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:17.597837    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:17.705413 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:17.705246    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:17.805776 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:17.805671    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:20.769790 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:20.769753    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:20.945824 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:20.945784    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:21.906528 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:21.906496    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:22.000960 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:22.000929    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv"
Sep 11 12:07:22.001152 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:22.001123    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:22.065720 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:22.065639    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36e69a19-0f39-41aa-b80e-e60b2b9cf3c7-istio-envoy\") pod \"36e69a19-0f39-41aa-b80e-e60b2b9cf3c7\" (UID: \"36e69a19-0f39-41aa-b80e-e60b2b9cf3c7\") "
Sep 11 12:07:22.065720 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:22.065682    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36e69a19-0f39-41aa-b80e-e60b2b9cf3c7-istio-token\") pod \"36e69a19-0f39-41aa-b80e-e60b2b9cf3c7\" (UID: \"36e69a19-0f39-41aa-b80e-e60b2b9cf3c7\") "
Sep 11 12:07:22.065720 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:22.065714    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36e69a19-0f39-41aa-b80e-e60b2b9cf3c7-workload-socket\") pod \"36e69a19-0f39-41aa-b80e-e60b2b9cf3c7\" (UID: \"36e69a19-0f39-41aa-b80e-e60b2b9cf3c7\") "
Sep 11 12:07:22.065996 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:22.065742    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36e69a19-0f39-41aa-b80e-e60b2b9cf3c7-istiod-ca-cert\") pod \"36e69a19-0f39-41aa-b80e-e60b2b9cf3c7\" (UID: \"36e69a19-0f39-41aa-b80e-e60b2b9cf3c7\") "
Sep 11 12:07:22.065996 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:22.065767    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36e69a19-0f39-41aa-b80e-e60b2b9cf3c7-workload-certs\") pod \"36e69a19-0f39-41aa-b80e-e60b2b9cf3c7\" (UID: \"36e69a19-0f39-41aa-b80e-e60b2b9cf3c7\") "
Sep 11 12:07:22.065996 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:22.065804    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36e69a19-0f39-41aa-b80e-e60b2b9cf3c7-istio-data\") pod \"36e69a19-0f39-41aa-b80e-e60b2b9cf3c7\" (UID: \"36e69a19-0f39-41aa-b80e-e60b2b9cf3c7\") "
Sep 11 12:07:22.065996 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:22.065832    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36e69a19-0f39-41aa-b80e-e60b2b9cf3c7-credential-socket\") pod \"36e69a19-0f39-41aa-b80e-e60b2b9cf3c7\" (UID: \"36e69a19-0f39-41aa-b80e-e60b2b9cf3c7\") "
Sep 11 12:07:22.065996 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:22.065867    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36e69a19-0f39-41aa-b80e-e60b2b9cf3c7-istio-podinfo\") pod \"36e69a19-0f39-41aa-b80e-e60b2b9cf3c7\" (UID: \"36e69a19-0f39-41aa-b80e-e60b2b9cf3c7\") "
Sep 11 12:07:22.065996 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:22.065940    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q8vdj\" (UniqueName: \"kubernetes.io/projected/36e69a19-0f39-41aa-b80e-e60b2b9cf3c7-kube-api-access-q8vdj\") pod \"36e69a19-0f39-41aa-b80e-e60b2b9cf3c7\" (UID: \"36e69a19-0f39-41aa-b80e-e60b2b9cf3c7\") "
Sep 11 12:07:22.066303 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:22.066170    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36e69a19-0f39-41aa-b80e-e60b2b9cf3c7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "36e69a19-0f39-41aa-b80e-e60b2b9cf3c7" (UID: "36e69a19-0f39-41aa-b80e-e60b2b9cf3c7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:07:22.066303 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:22.066198    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36e69a19-0f39-41aa-b80e-e60b2b9cf3c7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "36e69a19-0f39-41aa-b80e-e60b2b9cf3c7" (UID: "36e69a19-0f39-41aa-b80e-e60b2b9cf3c7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:07:22.066303 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:22.066185    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36e69a19-0f39-41aa-b80e-e60b2b9cf3c7-istio-data" (OuterVolumeSpecName: "istio-data") pod "36e69a19-0f39-41aa-b80e-e60b2b9cf3c7" (UID: "36e69a19-0f39-41aa-b80e-e60b2b9cf3c7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:07:22.066303 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:22.066255    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36e69a19-0f39-41aa-b80e-e60b2b9cf3c7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "36e69a19-0f39-41aa-b80e-e60b2b9cf3c7" (UID: "36e69a19-0f39-41aa-b80e-e60b2b9cf3c7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:07:22.066655 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:22.066369    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36e69a19-0f39-41aa-b80e-e60b2b9cf3c7-workload-socket\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:07:22.066655 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:22.066419    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36e69a19-0f39-41aa-b80e-e60b2b9cf3c7-istiod-ca-cert\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:07:22.066655 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:22.066436    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36e69a19-0f39-41aa-b80e-e60b2b9cf3c7-istio-data\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:07:22.066655 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:22.066448    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36e69a19-0f39-41aa-b80e-e60b2b9cf3c7-credential-socket\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:07:22.066655 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:22.066445    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36e69a19-0f39-41aa-b80e-e60b2b9cf3c7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "36e69a19-0f39-41aa-b80e-e60b2b9cf3c7" (UID: "36e69a19-0f39-41aa-b80e-e60b2b9cf3c7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:07:22.068050 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:22.068024    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36e69a19-0f39-41aa-b80e-e60b2b9cf3c7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "36e69a19-0f39-41aa-b80e-e60b2b9cf3c7" (UID: "36e69a19-0f39-41aa-b80e-e60b2b9cf3c7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:07:22.068050 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:22.068040    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36e69a19-0f39-41aa-b80e-e60b2b9cf3c7-istio-token" (OuterVolumeSpecName: "istio-token") pod "36e69a19-0f39-41aa-b80e-e60b2b9cf3c7" (UID: "36e69a19-0f39-41aa-b80e-e60b2b9cf3c7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:07:22.068443 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:22.068420    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/36e69a19-0f39-41aa-b80e-e60b2b9cf3c7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "36e69a19-0f39-41aa-b80e-e60b2b9cf3c7" (UID: "36e69a19-0f39-41aa-b80e-e60b2b9cf3c7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 12:07:22.068602 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:22.068579    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36e69a19-0f39-41aa-b80e-e60b2b9cf3c7-kube-api-access-q8vdj" (OuterVolumeSpecName: "kube-api-access-q8vdj") pod "36e69a19-0f39-41aa-b80e-e60b2b9cf3c7" (UID: "36e69a19-0f39-41aa-b80e-e60b2b9cf3c7"). InnerVolumeSpecName "kube-api-access-q8vdj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:07:22.167690 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:22.167649    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q8vdj\" (UniqueName: \"kubernetes.io/projected/36e69a19-0f39-41aa-b80e-e60b2b9cf3c7-kube-api-access-q8vdj\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:07:22.167690 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:22.167684    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36e69a19-0f39-41aa-b80e-e60b2b9cf3c7-istio-envoy\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:07:22.167690 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:22.167693    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36e69a19-0f39-41aa-b80e-e60b2b9cf3c7-istio-token\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:07:22.167690 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:22.167701    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36e69a19-0f39-41aa-b80e-e60b2b9cf3c7-workload-certs\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:07:22.167960 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:22.167709    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36e69a19-0f39-41aa-b80e-e60b2b9cf3c7-istio-podinfo\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:07:22.200224 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:22.200177    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:22.272356 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:22.272321    2579 generic.go:356] "Generic (PLEG): container finished" podID="36e69a19-0f39-41aa-b80e-e60b2b9cf3c7" containerID="c6587bc5c46ed78642d9cf1395f52b68f153af5e25cfeb65a33d1cfe68d081cf" exitCode=0
Sep 11 12:07:22.272541 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:22.272371    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv"
Sep 11 12:07:22.272541 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:22.272437    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv" event={"ID":"36e69a19-0f39-41aa-b80e-e60b2b9cf3c7","Type":"ContainerDied","Data":"c6587bc5c46ed78642d9cf1395f52b68f153af5e25cfeb65a33d1cfe68d081cf"}
Sep 11 12:07:22.272541 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:22.272498    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv" event={"ID":"36e69a19-0f39-41aa-b80e-e60b2b9cf3c7","Type":"ContainerDied","Data":"2398fed4df503a7aaa893138fe31f0a2de7e7e9287692e5275787e2481a4e6c7"}
Sep 11 12:07:22.272541 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:22.272526    2579 scope.go:117] "RemoveContainer" containerID="c6587bc5c46ed78642d9cf1395f52b68f153af5e25cfeb65a33d1cfe68d081cf"
Sep 11 12:07:22.282734 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:22.282714    2579 scope.go:117] "RemoveContainer" containerID="e5491aa364bf970cb3d5d9121e4da22b60df276ed7520131ef95cbeceb42b291"
Sep 11 12:07:22.291493 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:22.291467    2579 scope.go:117] "RemoveContainer" containerID="c6587bc5c46ed78642d9cf1395f52b68f153af5e25cfeb65a33d1cfe68d081cf"
Sep 11 12:07:22.291804 ip-10-0-142-145 kubenswrapper[2579]: E0911 12:07:22.291785    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6587bc5c46ed78642d9cf1395f52b68f153af5e25cfeb65a33d1cfe68d081cf\": container with ID starting with c6587bc5c46ed78642d9cf1395f52b68f153af5e25cfeb65a33d1cfe68d081cf not found: ID does not exist" containerID="c6587bc5c46ed78642d9cf1395f52b68f153af5e25cfeb65a33d1cfe68d081cf"
Sep 11 12:07:22.291860 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:22.291813    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6587bc5c46ed78642d9cf1395f52b68f153af5e25cfeb65a33d1cfe68d081cf"} err="failed to get container status \"c6587bc5c46ed78642d9cf1395f52b68f153af5e25cfeb65a33d1cfe68d081cf\": rpc error: code = NotFound desc = could not find container \"c6587bc5c46ed78642d9cf1395f52b68f153af5e25cfeb65a33d1cfe68d081cf\": container with ID starting with c6587bc5c46ed78642d9cf1395f52b68f153af5e25cfeb65a33d1cfe68d081cf not found: ID does not exist"
Sep 11 12:07:22.291860 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:22.291831    2579 scope.go:117] "RemoveContainer" containerID="e5491aa364bf970cb3d5d9121e4da22b60df276ed7520131ef95cbeceb42b291"
Sep 11 12:07:22.292106 ip-10-0-142-145 kubenswrapper[2579]: E0911 12:07:22.292091    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e5491aa364bf970cb3d5d9121e4da22b60df276ed7520131ef95cbeceb42b291\": container with ID starting with e5491aa364bf970cb3d5d9121e4da22b60df276ed7520131ef95cbeceb42b291 not found: ID does not exist" containerID="e5491aa364bf970cb3d5d9121e4da22b60df276ed7520131ef95cbeceb42b291"
Sep 11 12:07:22.292148 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:22.292111    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e5491aa364bf970cb3d5d9121e4da22b60df276ed7520131ef95cbeceb42b291"} err="failed to get container status \"e5491aa364bf970cb3d5d9121e4da22b60df276ed7520131ef95cbeceb42b291\": rpc error: code = NotFound desc = could not find container \"e5491aa364bf970cb3d5d9121e4da22b60df276ed7520131ef95cbeceb42b291\": container with ID starting with e5491aa364bf970cb3d5d9121e4da22b60df276ed7520131ef95cbeceb42b291 not found: ID does not exist"
Sep 11 12:07:22.296051 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:22.296022    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv"]
Sep 11 12:07:22.305876 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:22.305843    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-12d2a9-openshift-default-5844c946c4-8sblv"]
Sep 11 12:07:22.576854 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:22.576822    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36e69a19-0f39-41aa-b80e-e60b2b9cf3c7" path="/var/lib/kubelet/pods/36e69a19-0f39-41aa-b80e-e60b2b9cf3c7/volumes"
Sep 11 12:07:22.623341 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:22.623306    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-af081c-56499d888b-pg28q"]
Sep 11 12:07:22.623896 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:22.623870    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36e69a19-0f39-41aa-b80e-e60b2b9cf3c7" containerName="istio-proxy"
Sep 11 12:07:22.623896 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:22.623894    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="36e69a19-0f39-41aa-b80e-e60b2b9cf3c7" containerName="istio-proxy"
Sep 11 12:07:22.624090 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:22.623910    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36e69a19-0f39-41aa-b80e-e60b2b9cf3c7" containerName="istio-proxy"
Sep 11 12:07:22.624090 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:22.623920    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="36e69a19-0f39-41aa-b80e-e60b2b9cf3c7" containerName="istio-proxy"
Sep 11 12:07:22.624090 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:22.623944    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2418f3a0-c6e5-4a97-b33d-f5f62fd70741" containerName="maas-api"
Sep 11 12:07:22.624090 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:22.623951    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2418f3a0-c6e5-4a97-b33d-f5f62fd70741" containerName="maas-api"
Sep 11 12:07:22.624090 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:22.624035    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="36e69a19-0f39-41aa-b80e-e60b2b9cf3c7" containerName="istio-proxy"
Sep 11 12:07:22.624090 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:22.624046    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="36e69a19-0f39-41aa-b80e-e60b2b9cf3c7" containerName="istio-proxy"
Sep 11 12:07:22.624090 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:22.624054    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="2418f3a0-c6e5-4a97-b33d-f5f62fd70741" containerName="maas-api"
Sep 11 12:07:22.627320 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:22.627301    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-af081c-56499d888b-pg28q"
Sep 11 12:07:22.630075 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:22.630021    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-af081c\""
Sep 11 12:07:22.639224 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:22.639200    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-af081c-56499d888b-pg28q"]
Sep 11 12:07:22.773049 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:22.773000    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c9771a3-444a-4013-b989-10ab1e733615-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-af081c-56499d888b-pg28q\" (UID: \"8c9771a3-444a-4013-b989-10ab1e733615\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-af081c-56499d888b-pg28q"
Sep 11 12:07:22.773268 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:22.773162    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6l2m\" (UniqueName: \"kubernetes.io/projected/8c9771a3-444a-4013-b989-10ab1e733615-kube-api-access-k6l2m\") pod \"payload-pre-processing-e2e-shared-a-w3-af081c-56499d888b-pg28q\" (UID: \"8c9771a3-444a-4013-b989-10ab1e733615\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-af081c-56499d888b-pg28q"
Sep 11 12:07:22.874491 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:22.874459    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c9771a3-444a-4013-b989-10ab1e733615-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-af081c-56499d888b-pg28q\" (UID: \"8c9771a3-444a-4013-b989-10ab1e733615\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-af081c-56499d888b-pg28q"
Sep 11 12:07:22.874677 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:22.874553    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6l2m\" (UniqueName: \"kubernetes.io/projected/8c9771a3-444a-4013-b989-10ab1e733615-kube-api-access-k6l2m\") pod \"payload-pre-processing-e2e-shared-a-w3-af081c-56499d888b-pg28q\" (UID: \"8c9771a3-444a-4013-b989-10ab1e733615\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-af081c-56499d888b-pg28q"
Sep 11 12:07:22.875144 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:22.875122    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c9771a3-444a-4013-b989-10ab1e733615-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-af081c-56499d888b-pg28q\" (UID: \"8c9771a3-444a-4013-b989-10ab1e733615\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-af081c-56499d888b-pg28q"
Sep 11 12:07:22.883280 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:22.883248    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6l2m\" (UniqueName: \"kubernetes.io/projected/8c9771a3-444a-4013-b989-10ab1e733615-kube-api-access-k6l2m\") pod \"payload-pre-processing-e2e-shared-a-w3-af081c-56499d888b-pg28q\" (UID: \"8c9771a3-444a-4013-b989-10ab1e733615\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-af081c-56499d888b-pg28q"
Sep 11 12:07:22.938503 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:22.938461    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-af081c-56499d888b-pg28q"
Sep 11 12:07:23.210138 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:23.210111    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-af081c-56499d888b-pg28q"]
Sep 11 12:07:23.212614 ip-10-0-142-145 kubenswrapper[2579]: W0911 12:07:23.212586    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c9771a3_444a_4013_b989_10ab1e733615.slice/crio-deefaa25f2b52207077f4a409581820bbba2f944af9ede30944c222d827b96cb WatchSource:0}: Error finding container deefaa25f2b52207077f4a409581820bbba2f944af9ede30944c222d827b96cb: Status 404 returned error can't find the container with id deefaa25f2b52207077f4a409581820bbba2f944af9ede30944c222d827b96cb
Sep 11 12:07:23.281039 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:23.281002    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-af081c-56499d888b-pg28q" event={"ID":"8c9771a3-444a-4013-b989-10ab1e733615","Type":"ContainerStarted","Data":"deefaa25f2b52207077f4a409581820bbba2f944af9ede30944c222d827b96cb"}
Sep 11 12:07:23.409305 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:23.409267    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-24cfea9d-tv8nw"]
Sep 11 12:07:23.412416 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:23.412382    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-24cfea9d-tv8nw"
Sep 11 12:07:23.415626 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:23.415602    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9xkpf\""
Sep 11 12:07:23.416146 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:23.416126    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 12:07:23.421886 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:23.421860    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-24cfea9d-tv8nw"]
Sep 11 12:07:23.454425 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:23.454364    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:23.480287 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:23.480207    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/297659ea-e1a6-4910-801a-12c68c83555d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-24cfea9d-tv8nw\" (UID: \"297659ea-e1a6-4910-801a-12c68c83555d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-24cfea9d-tv8nw"
Sep 11 12:07:23.526228 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:23.526172    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:23.581155 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:23.581123    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/297659ea-e1a6-4910-801a-12c68c83555d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-24cfea9d-tv8nw\" (UID: \"297659ea-e1a6-4910-801a-12c68c83555d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-24cfea9d-tv8nw"
Sep 11 12:07:23.581734 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:23.581714    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/297659ea-e1a6-4910-801a-12c68c83555d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-24cfea9d-tv8nw\" (UID: \"297659ea-e1a6-4910-801a-12c68c83555d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-24cfea9d-tv8nw"
Sep 11 12:07:23.724467 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:23.724424    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-24cfea9d-tv8nw"
Sep 11 12:07:23.757817 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:23.757779    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:24.000498 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:24.000371    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-24cfea9d-tv8nw"]
Sep 11 12:07:24.002119 ip-10-0-142-145 kubenswrapper[2579]: W0911 12:07:24.002080    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod297659ea_e1a6_4910_801a_12c68c83555d.slice/crio-193aef126a74170c9e1ec9c318da0a11518c7eb5e1a182a4baaeb3e1ecb1e6e9 WatchSource:0}: Error finding container 193aef126a74170c9e1ec9c318da0a11518c7eb5e1a182a4baaeb3e1ecb1e6e9: Status 404 returned error can't find the container with id 193aef126a74170c9e1ec9c318da0a11518c7eb5e1a182a4baaeb3e1ecb1e6e9
Sep 11 12:07:24.288899 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:24.288862    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-24cfea9d-tv8nw" event={"ID":"297659ea-e1a6-4910-801a-12c68c83555d","Type":"ContainerStarted","Data":"193aef126a74170c9e1ec9c318da0a11518c7eb5e1a182a4baaeb3e1ecb1e6e9"}
Sep 11 12:07:24.290684 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:24.290650    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-af081c-56499d888b-pg28q" event={"ID":"8c9771a3-444a-4013-b989-10ab1e733615","Type":"ContainerStarted","Data":"3ccbb68e189052dee1600d07f429df1a656853d60d789a7ff3e854aa83d03576"}
Sep 11 12:07:24.290821 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:24.290803    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-af081c-56499d888b-pg28q"
Sep 11 12:07:24.311354 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:24.311294    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-af081c-56499d888b-pg28q" podStartSLOduration=2.31127974 podStartE2EDuration="2.31127974s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:07:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:07:24.30899322 +0000 UTC m=+1148.206434979" watchObservedRunningTime="2026-09-11 12:07:24.31127974 +0000 UTC m=+1148.208721481"
Sep 11 12:07:24.487311 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:24.487268    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:25.296937 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:25.296902    2579 generic.go:356] "Generic (PLEG): container finished" podID="297659ea-e1a6-4910-801a-12c68c83555d" containerID="2e55580364b381c0e4decb71b7f77c9348f7699798e5c6d7f57ea468e52e206a" exitCode=0
Sep 11 12:07:25.297362 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:25.296994    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-24cfea9d-tv8nw" event={"ID":"297659ea-e1a6-4910-801a-12c68c83555d","Type":"ContainerDied","Data":"2e55580364b381c0e4decb71b7f77c9348f7699798e5c6d7f57ea468e52e206a"}
Sep 11 12:07:26.443497 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:26.443468    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-24cfea9d-tv8nw"
Sep 11 12:07:26.469730 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:26.469686    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:07:26.512990 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:26.512958    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/297659ea-e1a6-4910-801a-12c68c83555d-maas-api-service-ca\") pod \"297659ea-e1a6-4910-801a-12c68c83555d\" (UID: \"297659ea-e1a6-4910-801a-12c68c83555d\") "
Sep 11 12:07:26.513308 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:26.513287    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/297659ea-e1a6-4910-801a-12c68c83555d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "297659ea-e1a6-4910-801a-12c68c83555d" (UID: "297659ea-e1a6-4910-801a-12c68c83555d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:07:26.614672 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:26.614624    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/297659ea-e1a6-4910-801a-12c68c83555d-maas-api-service-ca\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:07:27.308834 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:27.308799    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-24cfea9d-tv8nw" event={"ID":"297659ea-e1a6-4910-801a-12c68c83555d","Type":"ContainerDied","Data":"193aef126a74170c9e1ec9c318da0a11518c7eb5e1a182a4baaeb3e1ecb1e6e9"}
Sep 11 12:07:27.309018 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:27.308853    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="193aef126a74170c9e1ec9c318da0a11518c7eb5e1a182a4baaeb3e1ecb1e6e9"
Sep 11 12:07:27.309018 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:27.308811    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-24cfea9d-tv8nw"
Sep 11 12:07:33.585590 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:33.585555    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:34.256146 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:34.256107    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:34.308377 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:34.308340    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-24cfea9d-7b69fc9c8b-bh8ds"]
Sep 11 12:07:34.308599 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:34.308576    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-24cfea9d-7b69fc9c8b-bh8ds" podUID="80ab63cd-18b4-42fd-ae10-22f7dac14c49" containerName="payload-pre-processing" containerID="cri-o://e8c3d463c969eaeead544c5b78cf98cf15e37e8d9925411f57f4ab534c59a3e8" gracePeriod=30
Sep 11 12:07:34.356903 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:34.356869    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:34.463777 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:34.463743    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:34.566284 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:34.566257    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-24cfea9d-7b69fc9c8b-bh8ds"
Sep 11 12:07:34.579810 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:34.579778    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:34.601978 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:34.600985    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-24cfea9d-tv8nw"]
Sep 11 12:07:34.609559 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:34.609526    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-24cfea9d-tv8nw"]
Sep 11 12:07:34.694363 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:34.694325    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80ab63cd-18b4-42fd-ae10-22f7dac14c49-plugins-config-volume\") pod \"80ab63cd-18b4-42fd-ae10-22f7dac14c49\" (UID: \"80ab63cd-18b4-42fd-ae10-22f7dac14c49\") "
Sep 11 12:07:34.694574 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:34.694446    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fss5n\" (UniqueName: \"kubernetes.io/projected/80ab63cd-18b4-42fd-ae10-22f7dac14c49-kube-api-access-fss5n\") pod \"80ab63cd-18b4-42fd-ae10-22f7dac14c49\" (UID: \"80ab63cd-18b4-42fd-ae10-22f7dac14c49\") "
Sep 11 12:07:34.694821 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:34.694789    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80ab63cd-18b4-42fd-ae10-22f7dac14c49-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "80ab63cd-18b4-42fd-ae10-22f7dac14c49" (UID: "80ab63cd-18b4-42fd-ae10-22f7dac14c49"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:07:34.695061 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:34.695034    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80ab63cd-18b4-42fd-ae10-22f7dac14c49-plugins-config-volume\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:07:34.696696 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:34.696665    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80ab63cd-18b4-42fd-ae10-22f7dac14c49-kube-api-access-fss5n" (OuterVolumeSpecName: "kube-api-access-fss5n") pod "80ab63cd-18b4-42fd-ae10-22f7dac14c49" (UID: "80ab63cd-18b4-42fd-ae10-22f7dac14c49"). InnerVolumeSpecName "kube-api-access-fss5n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:07:34.732970 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:34.732933    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:34.795792 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:34.795701    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fss5n\" (UniqueName: \"kubernetes.io/projected/80ab63cd-18b4-42fd-ae10-22f7dac14c49-kube-api-access-fss5n\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:07:34.995835 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:34.995794    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:35.299805 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:35.299775    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-af081c-56499d888b-pg28q"
Sep 11 12:07:35.352950 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:35.352910    2579 generic.go:356] "Generic (PLEG): container finished" podID="80ab63cd-18b4-42fd-ae10-22f7dac14c49" containerID="e8c3d463c969eaeead544c5b78cf98cf15e37e8d9925411f57f4ab534c59a3e8" exitCode=0
Sep 11 12:07:35.353134 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:35.352963    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-24cfea9d-7b69fc9c8b-bh8ds"
Sep 11 12:07:35.353134 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:35.352994    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-24cfea9d-7b69fc9c8b-bh8ds" event={"ID":"80ab63cd-18b4-42fd-ae10-22f7dac14c49","Type":"ContainerDied","Data":"e8c3d463c969eaeead544c5b78cf98cf15e37e8d9925411f57f4ab534c59a3e8"}
Sep 11 12:07:35.353134 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:35.353044    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-24cfea9d-7b69fc9c8b-bh8ds" event={"ID":"80ab63cd-18b4-42fd-ae10-22f7dac14c49","Type":"ContainerDied","Data":"ecad6b22eb43af4b447db12a25e7e8f5bb0d63ace295d9a86e22a0fa52f16c4c"}
Sep 11 12:07:35.353134 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:35.353070    2579 scope.go:117] "RemoveContainer" containerID="e8c3d463c969eaeead544c5b78cf98cf15e37e8d9925411f57f4ab534c59a3e8"
Sep 11 12:07:35.362420 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:35.362385    2579 scope.go:117] "RemoveContainer" containerID="e8c3d463c969eaeead544c5b78cf98cf15e37e8d9925411f57f4ab534c59a3e8"
Sep 11 12:07:35.362750 ip-10-0-142-145 kubenswrapper[2579]: E0911 12:07:35.362729    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e8c3d463c969eaeead544c5b78cf98cf15e37e8d9925411f57f4ab534c59a3e8\": container with ID starting with e8c3d463c969eaeead544c5b78cf98cf15e37e8d9925411f57f4ab534c59a3e8 not found: ID does not exist" containerID="e8c3d463c969eaeead544c5b78cf98cf15e37e8d9925411f57f4ab534c59a3e8"
Sep 11 12:07:35.362809 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:35.362758    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8c3d463c969eaeead544c5b78cf98cf15e37e8d9925411f57f4ab534c59a3e8"} err="failed to get container status \"e8c3d463c969eaeead544c5b78cf98cf15e37e8d9925411f57f4ab534c59a3e8\": rpc error: code = NotFound desc = could not find container \"e8c3d463c969eaeead544c5b78cf98cf15e37e8d9925411f57f4ab534c59a3e8\": container with ID starting with e8c3d463c969eaeead544c5b78cf98cf15e37e8d9925411f57f4ab534c59a3e8 not found: ID does not exist"
Sep 11 12:07:35.397920 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:35.397887    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-24cfea9d-7b69fc9c8b-bh8ds"]
Sep 11 12:07:35.411914 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:35.411885    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-24cfea9d-7b69fc9c8b-bh8ds"]
Sep 11 12:07:36.576171 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:36.576136    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="297659ea-e1a6-4910-801a-12c68c83555d" path="/var/lib/kubelet/pods/297659ea-e1a6-4910-801a-12c68c83555d/volumes"
Sep 11 12:07:36.576582 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:36.576546    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="80ab63cd-18b4-42fd-ae10-22f7dac14c49" path="/var/lib/kubelet/pods/80ab63cd-18b4-42fd-ae10-22f7dac14c49/volumes"
Sep 11 12:07:37.579199 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:37.579161    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:07:41.241749 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:41.241672    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:41.394439 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:41.394379    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:41.560161 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:41.560057    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9"]
Sep 11 12:07:41.560381 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:41.560337    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9" podUID="16aaa4c7-eb22-436e-ab60-12d4c78a732e" containerName="istio-proxy" containerID="cri-o://062315248de70e9791b5fbebd229ad8a027420b4b1cf998f20c8d1052fa68929" gracePeriod=30
Sep 11 12:07:41.567538 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:41.567509    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:41.869285 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:41.869249    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:41.971957 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:41.971915    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:42.114558 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:42.114525    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:43.086292 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:43.086248    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:45.608639 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:45.608601    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:45.894457 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:45.894366    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:46.193030 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:46.192936    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65"]
Sep 11 12:07:46.193484 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:46.193465    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80ab63cd-18b4-42fd-ae10-22f7dac14c49" containerName="payload-pre-processing"
Sep 11 12:07:46.193591 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:46.193487    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="80ab63cd-18b4-42fd-ae10-22f7dac14c49" containerName="payload-pre-processing"
Sep 11 12:07:46.193591 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:46.193501    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="297659ea-e1a6-4910-801a-12c68c83555d" containerName="revoke-keys"
Sep 11 12:07:46.193591 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:46.193509    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="297659ea-e1a6-4910-801a-12c68c83555d" containerName="revoke-keys"
Sep 11 12:07:46.193748 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:46.193631    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="297659ea-e1a6-4910-801a-12c68c83555d" containerName="revoke-keys"
Sep 11 12:07:46.193748 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:46.193646    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="80ab63cd-18b4-42fd-ae10-22f7dac14c49" containerName="payload-pre-processing"
Sep 11 12:07:46.197218 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:46.197197    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65"
Sep 11 12:07:46.200247 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:46.200225    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-d844cc-openshift-default-dockercfg-2wzm8\""
Sep 11 12:07:46.213767 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:46.213730    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65"]
Sep 11 12:07:46.227990 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:46.227955    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:46.301452 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:46.301380    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/45e86e88-5ccd-4c4c-8d31-8350e844f996-istio-podinfo\") pod \"e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65\" (UID: \"45e86e88-5ccd-4c4c-8d31-8350e844f996\") " pod="openshift-ingress/e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65"
Sep 11 12:07:46.301452 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:46.301448    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/45e86e88-5ccd-4c4c-8d31-8350e844f996-workload-certs\") pod \"e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65\" (UID: \"45e86e88-5ccd-4c4c-8d31-8350e844f996\") " pod="openshift-ingress/e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65"
Sep 11 12:07:46.301719 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:46.301470    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ptwz\" (UniqueName: \"kubernetes.io/projected/45e86e88-5ccd-4c4c-8d31-8350e844f996-kube-api-access-4ptwz\") pod \"e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65\" (UID: \"45e86e88-5ccd-4c4c-8d31-8350e844f996\") " pod="openshift-ingress/e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65"
Sep 11 12:07:46.301719 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:46.301604    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/45e86e88-5ccd-4c4c-8d31-8350e844f996-istiod-ca-cert\") pod \"e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65\" (UID: \"45e86e88-5ccd-4c4c-8d31-8350e844f996\") " pod="openshift-ingress/e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65"
Sep 11 12:07:46.301719 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:46.301658    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/45e86e88-5ccd-4c4c-8d31-8350e844f996-workload-socket\") pod \"e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65\" (UID: \"45e86e88-5ccd-4c4c-8d31-8350e844f996\") " pod="openshift-ingress/e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65"
Sep 11 12:07:46.301719 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:46.301711    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/45e86e88-5ccd-4c4c-8d31-8350e844f996-credential-socket\") pod \"e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65\" (UID: \"45e86e88-5ccd-4c4c-8d31-8350e844f996\") " pod="openshift-ingress/e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65"
Sep 11 12:07:46.301888 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:46.301744    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/45e86e88-5ccd-4c4c-8d31-8350e844f996-istio-token\") pod \"e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65\" (UID: \"45e86e88-5ccd-4c4c-8d31-8350e844f996\") " pod="openshift-ingress/e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65"
Sep 11 12:07:46.301888 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:46.301765    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/45e86e88-5ccd-4c4c-8d31-8350e844f996-istio-data\") pod \"e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65\" (UID: \"45e86e88-5ccd-4c4c-8d31-8350e844f996\") " pod="openshift-ingress/e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65"
Sep 11 12:07:46.301888 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:46.301791    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/45e86e88-5ccd-4c4c-8d31-8350e844f996-istio-envoy\") pod \"e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65\" (UID: \"45e86e88-5ccd-4c4c-8d31-8350e844f996\") " pod="openshift-ingress/e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65"
Sep 11 12:07:46.403153 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:46.403120    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ptwz\" (UniqueName: \"kubernetes.io/projected/45e86e88-5ccd-4c4c-8d31-8350e844f996-kube-api-access-4ptwz\") pod \"e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65\" (UID: \"45e86e88-5ccd-4c4c-8d31-8350e844f996\") " pod="openshift-ingress/e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65"
Sep 11 12:07:46.403346 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:46.403204    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/45e86e88-5ccd-4c4c-8d31-8350e844f996-istiod-ca-cert\") pod \"e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65\" (UID: \"45e86e88-5ccd-4c4c-8d31-8350e844f996\") " pod="openshift-ingress/e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65"
Sep 11 12:07:46.403346 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:46.403234    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/45e86e88-5ccd-4c4c-8d31-8350e844f996-workload-socket\") pod \"e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65\" (UID: \"45e86e88-5ccd-4c4c-8d31-8350e844f996\") " pod="openshift-ingress/e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65"
Sep 11 12:07:46.403346 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:46.403256    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/45e86e88-5ccd-4c4c-8d31-8350e844f996-credential-socket\") pod \"e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65\" (UID: \"45e86e88-5ccd-4c4c-8d31-8350e844f996\") " pod="openshift-ingress/e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65"
Sep 11 12:07:46.403346 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:46.403279    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/45e86e88-5ccd-4c4c-8d31-8350e844f996-istio-token\") pod \"e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65\" (UID: \"45e86e88-5ccd-4c4c-8d31-8350e844f996\") " pod="openshift-ingress/e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65"
Sep 11 12:07:46.403346 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:46.403305    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/45e86e88-5ccd-4c4c-8d31-8350e844f996-istio-data\") pod \"e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65\" (UID: \"45e86e88-5ccd-4c4c-8d31-8350e844f996\") " pod="openshift-ingress/e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65"
Sep 11 12:07:46.403663 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:46.403457    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/45e86e88-5ccd-4c4c-8d31-8350e844f996-istio-envoy\") pod \"e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65\" (UID: \"45e86e88-5ccd-4c4c-8d31-8350e844f996\") " pod="openshift-ingress/e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65"
Sep 11 12:07:46.403663 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:46.403565    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/45e86e88-5ccd-4c4c-8d31-8350e844f996-istio-podinfo\") pod \"e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65\" (UID: \"45e86e88-5ccd-4c4c-8d31-8350e844f996\") " pod="openshift-ingress/e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65"
Sep 11 12:07:46.403663 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:46.403590    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/45e86e88-5ccd-4c4c-8d31-8350e844f996-workload-certs\") pod \"e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65\" (UID: \"45e86e88-5ccd-4c4c-8d31-8350e844f996\") " pod="openshift-ingress/e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65"
Sep 11 12:07:46.403892 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:46.403698    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/45e86e88-5ccd-4c4c-8d31-8350e844f996-workload-socket\") pod \"e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65\" (UID: \"45e86e88-5ccd-4c4c-8d31-8350e844f996\") " pod="openshift-ingress/e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65"
Sep 11 12:07:46.403892 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:46.403874    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/45e86e88-5ccd-4c4c-8d31-8350e844f996-credential-socket\") pod \"e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65\" (UID: \"45e86e88-5ccd-4c4c-8d31-8350e844f996\") " pod="openshift-ingress/e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65"
Sep 11 12:07:46.404013 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:46.403932    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/45e86e88-5ccd-4c4c-8d31-8350e844f996-istio-data\") pod \"e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65\" (UID: \"45e86e88-5ccd-4c4c-8d31-8350e844f996\") " pod="openshift-ingress/e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65"
Sep 11 12:07:46.404013 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:46.403944    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/45e86e88-5ccd-4c4c-8d31-8350e844f996-workload-certs\") pod \"e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65\" (UID: \"45e86e88-5ccd-4c4c-8d31-8350e844f996\") " pod="openshift-ingress/e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65"
Sep 11 12:07:46.404144 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:46.404122    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/45e86e88-5ccd-4c4c-8d31-8350e844f996-istiod-ca-cert\") pod \"e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65\" (UID: \"45e86e88-5ccd-4c4c-8d31-8350e844f996\") " pod="openshift-ingress/e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65"
Sep 11 12:07:46.405731 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:46.405712    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/45e86e88-5ccd-4c4c-8d31-8350e844f996-istio-envoy\") pod \"e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65\" (UID: \"45e86e88-5ccd-4c4c-8d31-8350e844f996\") " pod="openshift-ingress/e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65"
Sep 11 12:07:46.406007 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:46.405989    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/45e86e88-5ccd-4c4c-8d31-8350e844f996-istio-podinfo\") pod \"e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65\" (UID: \"45e86e88-5ccd-4c4c-8d31-8350e844f996\") " pod="openshift-ingress/e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65"
Sep 11 12:07:46.417336 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:46.417305    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ptwz\" (UniqueName: \"kubernetes.io/projected/45e86e88-5ccd-4c4c-8d31-8350e844f996-kube-api-access-4ptwz\") pod \"e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65\" (UID: \"45e86e88-5ccd-4c4c-8d31-8350e844f996\") " pod="openshift-ingress/e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65"
Sep 11 12:07:46.427445 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:46.427367    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/45e86e88-5ccd-4c4c-8d31-8350e844f996-istio-token\") pod \"e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65\" (UID: \"45e86e88-5ccd-4c4c-8d31-8350e844f996\") " pod="openshift-ingress/e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65"
Sep 11 12:07:46.459607 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:46.459524    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:46.507360 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:46.507323    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65"
Sep 11 12:07:46.789108 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:46.789082    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9"
Sep 11 12:07:46.800698 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:46.800673    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65"]
Sep 11 12:07:46.802799 ip-10-0-142-145 kubenswrapper[2579]: W0911 12:07:46.802775    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45e86e88_5ccd_4c4c_8d31_8350e844f996.slice/crio-fd7fca3f9d12000bc1c3b8e2724b61d158768b802aca49235fab53d6f6f67410 WatchSource:0}: Error finding container fd7fca3f9d12000bc1c3b8e2724b61d158768b802aca49235fab53d6f6f67410: Status 404 returned error can't find the container with id fd7fca3f9d12000bc1c3b8e2724b61d158768b802aca49235fab53d6f6f67410
Sep 11 12:07:46.804970 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:46.804938    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 12:07:46.805068 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:46.804998    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 12:07:46.805068 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:46.805024    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 12:07:46.908082 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:46.908048    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/16aaa4c7-eb22-436e-ab60-12d4c78a732e-istio-envoy\") pod \"16aaa4c7-eb22-436e-ab60-12d4c78a732e\" (UID: \"16aaa4c7-eb22-436e-ab60-12d4c78a732e\") "
Sep 11 12:07:46.908266 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:46.908178    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/16aaa4c7-eb22-436e-ab60-12d4c78a732e-workload-socket\") pod \"16aaa4c7-eb22-436e-ab60-12d4c78a732e\" (UID: \"16aaa4c7-eb22-436e-ab60-12d4c78a732e\") "
Sep 11 12:07:46.908266 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:46.908254    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/16aaa4c7-eb22-436e-ab60-12d4c78a732e-workload-certs\") pod \"16aaa4c7-eb22-436e-ab60-12d4c78a732e\" (UID: \"16aaa4c7-eb22-436e-ab60-12d4c78a732e\") "
Sep 11 12:07:46.908427 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:46.908311    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/16aaa4c7-eb22-436e-ab60-12d4c78a732e-istio-data\") pod \"16aaa4c7-eb22-436e-ab60-12d4c78a732e\" (UID: \"16aaa4c7-eb22-436e-ab60-12d4c78a732e\") "
Sep 11 12:07:46.908427 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:46.908342    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/16aaa4c7-eb22-436e-ab60-12d4c78a732e-istio-token\") pod \"16aaa4c7-eb22-436e-ab60-12d4c78a732e\" (UID: \"16aaa4c7-eb22-436e-ab60-12d4c78a732e\") "
Sep 11 12:07:46.908546 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:46.908387    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/16aaa4c7-eb22-436e-ab60-12d4c78a732e-istio-podinfo\") pod \"16aaa4c7-eb22-436e-ab60-12d4c78a732e\" (UID: \"16aaa4c7-eb22-436e-ab60-12d4c78a732e\") "
Sep 11 12:07:46.908546 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:46.908463    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/16aaa4c7-eb22-436e-ab60-12d4c78a732e-credential-socket\") pod \"16aaa4c7-eb22-436e-ab60-12d4c78a732e\" (UID: \"16aaa4c7-eb22-436e-ab60-12d4c78a732e\") "
Sep 11 12:07:46.908546 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:46.908501    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s4mz5\" (UniqueName: \"kubernetes.io/projected/16aaa4c7-eb22-436e-ab60-12d4c78a732e-kube-api-access-s4mz5\") pod \"16aaa4c7-eb22-436e-ab60-12d4c78a732e\" (UID: \"16aaa4c7-eb22-436e-ab60-12d4c78a732e\") "
Sep 11 12:07:46.908546 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:46.908533    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/16aaa4c7-eb22-436e-ab60-12d4c78a732e-istiod-ca-cert\") pod \"16aaa4c7-eb22-436e-ab60-12d4c78a732e\" (UID: \"16aaa4c7-eb22-436e-ab60-12d4c78a732e\") "
Sep 11 12:07:46.908747 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:46.908627    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16aaa4c7-eb22-436e-ab60-12d4c78a732e-istio-data" (OuterVolumeSpecName: "istio-data") pod "16aaa4c7-eb22-436e-ab60-12d4c78a732e" (UID: "16aaa4c7-eb22-436e-ab60-12d4c78a732e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:07:46.908926 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:46.908907    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/16aaa4c7-eb22-436e-ab60-12d4c78a732e-istio-data\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:07:46.909309 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:46.909222    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16aaa4c7-eb22-436e-ab60-12d4c78a732e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "16aaa4c7-eb22-436e-ab60-12d4c78a732e" (UID: "16aaa4c7-eb22-436e-ab60-12d4c78a732e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:07:46.909422 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:46.909372    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16aaa4c7-eb22-436e-ab60-12d4c78a732e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "16aaa4c7-eb22-436e-ab60-12d4c78a732e" (UID: "16aaa4c7-eb22-436e-ab60-12d4c78a732e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:07:46.909568 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:46.909544    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16aaa4c7-eb22-436e-ab60-12d4c78a732e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "16aaa4c7-eb22-436e-ab60-12d4c78a732e" (UID: "16aaa4c7-eb22-436e-ab60-12d4c78a732e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:07:46.909661 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:46.909630    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16aaa4c7-eb22-436e-ab60-12d4c78a732e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "16aaa4c7-eb22-436e-ab60-12d4c78a732e" (UID: "16aaa4c7-eb22-436e-ab60-12d4c78a732e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:07:46.910960 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:46.910935    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16aaa4c7-eb22-436e-ab60-12d4c78a732e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "16aaa4c7-eb22-436e-ab60-12d4c78a732e" (UID: "16aaa4c7-eb22-436e-ab60-12d4c78a732e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:07:46.911193 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:46.911141    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16aaa4c7-eb22-436e-ab60-12d4c78a732e-kube-api-access-s4mz5" (OuterVolumeSpecName: "kube-api-access-s4mz5") pod "16aaa4c7-eb22-436e-ab60-12d4c78a732e" (UID: "16aaa4c7-eb22-436e-ab60-12d4c78a732e"). InnerVolumeSpecName "kube-api-access-s4mz5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:07:46.911193 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:46.911158    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/16aaa4c7-eb22-436e-ab60-12d4c78a732e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "16aaa4c7-eb22-436e-ab60-12d4c78a732e" (UID: "16aaa4c7-eb22-436e-ab60-12d4c78a732e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 12:07:46.911744 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:46.911728    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16aaa4c7-eb22-436e-ab60-12d4c78a732e-istio-token" (OuterVolumeSpecName: "istio-token") pod "16aaa4c7-eb22-436e-ab60-12d4c78a732e" (UID: "16aaa4c7-eb22-436e-ab60-12d4c78a732e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:07:47.009859 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:47.009776    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/16aaa4c7-eb22-436e-ab60-12d4c78a732e-istio-envoy\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:07:47.009859 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:47.009808    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/16aaa4c7-eb22-436e-ab60-12d4c78a732e-workload-socket\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:07:47.009859 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:47.009819    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/16aaa4c7-eb22-436e-ab60-12d4c78a732e-workload-certs\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:07:47.009859 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:47.009827    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/16aaa4c7-eb22-436e-ab60-12d4c78a732e-istio-token\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:07:47.009859 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:47.009835    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/16aaa4c7-eb22-436e-ab60-12d4c78a732e-istio-podinfo\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:07:47.009859 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:47.009842    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/16aaa4c7-eb22-436e-ab60-12d4c78a732e-credential-socket\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:07:47.009859 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:47.009856    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s4mz5\" (UniqueName: \"kubernetes.io/projected/16aaa4c7-eb22-436e-ab60-12d4c78a732e-kube-api-access-s4mz5\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:07:47.010180 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:47.009868    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/16aaa4c7-eb22-436e-ab60-12d4c78a732e-istiod-ca-cert\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:07:47.419568 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:47.419535    2579 generic.go:356] "Generic (PLEG): container finished" podID="16aaa4c7-eb22-436e-ab60-12d4c78a732e" containerID="062315248de70e9791b5fbebd229ad8a027420b4b1cf998f20c8d1052fa68929" exitCode=0
Sep 11 12:07:47.419684 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:47.419599    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9"
Sep 11 12:07:47.419684 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:47.419628    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9" event={"ID":"16aaa4c7-eb22-436e-ab60-12d4c78a732e","Type":"ContainerDied","Data":"062315248de70e9791b5fbebd229ad8a027420b4b1cf998f20c8d1052fa68929"}
Sep 11 12:07:47.419789 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:47.419701    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9" event={"ID":"16aaa4c7-eb22-436e-ab60-12d4c78a732e","Type":"ContainerDied","Data":"bc416b4a63294a6b165986930bf95fa845c4916b456cf869e0fe21500a811c40"}
Sep 11 12:07:47.419789 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:47.419732    2579 scope.go:117] "RemoveContainer" containerID="062315248de70e9791b5fbebd229ad8a027420b4b1cf998f20c8d1052fa68929"
Sep 11 12:07:47.421711 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:47.421690    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65" event={"ID":"45e86e88-5ccd-4c4c-8d31-8350e844f996","Type":"ContainerStarted","Data":"8ddbae16db110bb06d24ae0afb63573f49c1581c4f9550ba5ccd2292f86c5c2c"}
Sep 11 12:07:47.421829 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:47.421733    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65" event={"ID":"45e86e88-5ccd-4c4c-8d31-8350e844f996","Type":"ContainerStarted","Data":"fd7fca3f9d12000bc1c3b8e2724b61d158768b802aca49235fab53d6f6f67410"}
Sep 11 12:07:47.429315 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:47.429291    2579 scope.go:117] "RemoveContainer" containerID="c65b4c5b32b567b61eac3d928d7db6fd2253a21cbec6c0cbc960f21f5105f219"
Sep 11 12:07:47.437536 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:47.437519    2579 scope.go:117] "RemoveContainer" containerID="062315248de70e9791b5fbebd229ad8a027420b4b1cf998f20c8d1052fa68929"
Sep 11 12:07:47.437822 ip-10-0-142-145 kubenswrapper[2579]: E0911 12:07:47.437803    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"062315248de70e9791b5fbebd229ad8a027420b4b1cf998f20c8d1052fa68929\": container with ID starting with 062315248de70e9791b5fbebd229ad8a027420b4b1cf998f20c8d1052fa68929 not found: ID does not exist" containerID="062315248de70e9791b5fbebd229ad8a027420b4b1cf998f20c8d1052fa68929"
Sep 11 12:07:47.437880 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:47.437830    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"062315248de70e9791b5fbebd229ad8a027420b4b1cf998f20c8d1052fa68929"} err="failed to get container status \"062315248de70e9791b5fbebd229ad8a027420b4b1cf998f20c8d1052fa68929\": rpc error: code = NotFound desc = could not find container \"062315248de70e9791b5fbebd229ad8a027420b4b1cf998f20c8d1052fa68929\": container with ID starting with 062315248de70e9791b5fbebd229ad8a027420b4b1cf998f20c8d1052fa68929 not found: ID does not exist"
Sep 11 12:07:47.437880 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:47.437848    2579 scope.go:117] "RemoveContainer" containerID="c65b4c5b32b567b61eac3d928d7db6fd2253a21cbec6c0cbc960f21f5105f219"
Sep 11 12:07:47.438105 ip-10-0-142-145 kubenswrapper[2579]: E0911 12:07:47.438091    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c65b4c5b32b567b61eac3d928d7db6fd2253a21cbec6c0cbc960f21f5105f219\": container with ID starting with c65b4c5b32b567b61eac3d928d7db6fd2253a21cbec6c0cbc960f21f5105f219 not found: ID does not exist" containerID="c65b4c5b32b567b61eac3d928d7db6fd2253a21cbec6c0cbc960f21f5105f219"
Sep 11 12:07:47.438139 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:47.438109    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c65b4c5b32b567b61eac3d928d7db6fd2253a21cbec6c0cbc960f21f5105f219"} err="failed to get container status \"c65b4c5b32b567b61eac3d928d7db6fd2253a21cbec6c0cbc960f21f5105f219\": rpc error: code = NotFound desc = could not find container \"c65b4c5b32b567b61eac3d928d7db6fd2253a21cbec6c0cbc960f21f5105f219\": container with ID starting with c65b4c5b32b567b61eac3d928d7db6fd2253a21cbec6c0cbc960f21f5105f219 not found: ID does not exist"
Sep 11 12:07:47.450931 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:47.450885    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65" podStartSLOduration=1.450871702 podStartE2EDuration="1.450871702s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:07:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:07:47.448589069 +0000 UTC m=+1171.346030838" watchObservedRunningTime="2026-09-11 12:07:47.450871702 +0000 UTC m=+1171.348313446"
Sep 11 12:07:47.463262 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:47.463232    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9"]
Sep 11 12:07:47.469188 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:47.469157    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-24cfea9d-openshift-default-b56fdd8c6-276g9"]
Sep 11 12:07:47.508334 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:47.508296    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65"
Sep 11 12:07:47.513855 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:47.513827    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65"
Sep 11 12:07:48.427960 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:48.427925    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65"
Sep 11 12:07:48.429032 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:48.429015    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65"
Sep 11 12:07:48.577458 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:48.577424    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16aaa4c7-eb22-436e-ab60-12d4c78a732e" path="/var/lib/kubelet/pods/16aaa4c7-eb22-436e-ab60-12d4c78a732e/volumes"
Sep 11 12:07:49.138791 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:49.138756    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:49.243628 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:49.243592    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:49.450119 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:49.450031    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:49.529858 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:49.529816    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:50.916782 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:50.916748    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:51.017019 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:51.016978    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:51.319104 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:51.319016    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:51.421299 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:51.421261    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:51.449560 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:51.449528    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-d844cc-6c8446bb85-frh5c"]
Sep 11 12:07:51.449940 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:51.449928    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16aaa4c7-eb22-436e-ab60-12d4c78a732e" containerName="istio-proxy"
Sep 11 12:07:51.449999 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:51.449942    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="16aaa4c7-eb22-436e-ab60-12d4c78a732e" containerName="istio-proxy"
Sep 11 12:07:51.449999 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:51.449953    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16aaa4c7-eb22-436e-ab60-12d4c78a732e" containerName="istio-proxy"
Sep 11 12:07:51.449999 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:51.449959    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="16aaa4c7-eb22-436e-ab60-12d4c78a732e" containerName="istio-proxy"
Sep 11 12:07:51.450090 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:51.450022    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="16aaa4c7-eb22-436e-ab60-12d4c78a732e" containerName="istio-proxy"
Sep 11 12:07:51.452916 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:51.452900    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-d844cc-6c8446bb85-frh5c"
Sep 11 12:07:51.455656 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:51.455633    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-d844cc\""
Sep 11 12:07:51.455793 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:51.455777    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-d844cc-dockercfg-n24j9\""
Sep 11 12:07:51.465837 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:51.465790    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-d844cc-6c8446bb85-frh5c"]
Sep 11 12:07:51.556357 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:51.556324    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntq9s\" (UniqueName: \"kubernetes.io/projected/fa552843-e4cf-447a-a25d-288eb7f9660a-kube-api-access-ntq9s\") pod \"payload-processing-e2e-shared-b-w3-d844cc-6c8446bb85-frh5c\" (UID: \"fa552843-e4cf-447a-a25d-288eb7f9660a\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-d844cc-6c8446bb85-frh5c"
Sep 11 12:07:51.556589 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:51.556438    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa552843-e4cf-447a-a25d-288eb7f9660a-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-d844cc-6c8446bb85-frh5c\" (UID: \"fa552843-e4cf-447a-a25d-288eb7f9660a\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-d844cc-6c8446bb85-frh5c"
Sep 11 12:07:51.657236 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:51.657192    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa552843-e4cf-447a-a25d-288eb7f9660a-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-d844cc-6c8446bb85-frh5c\" (UID: \"fa552843-e4cf-447a-a25d-288eb7f9660a\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-d844cc-6c8446bb85-frh5c"
Sep 11 12:07:51.657444 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:51.657283    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntq9s\" (UniqueName: \"kubernetes.io/projected/fa552843-e4cf-447a-a25d-288eb7f9660a-kube-api-access-ntq9s\") pod \"payload-processing-e2e-shared-b-w3-d844cc-6c8446bb85-frh5c\" (UID: \"fa552843-e4cf-447a-a25d-288eb7f9660a\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-d844cc-6c8446bb85-frh5c"
Sep 11 12:07:51.658019 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:51.657997    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa552843-e4cf-447a-a25d-288eb7f9660a-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-d844cc-6c8446bb85-frh5c\" (UID: \"fa552843-e4cf-447a-a25d-288eb7f9660a\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-d844cc-6c8446bb85-frh5c"
Sep 11 12:07:51.666995 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:51.666959    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntq9s\" (UniqueName: \"kubernetes.io/projected/fa552843-e4cf-447a-a25d-288eb7f9660a-kube-api-access-ntq9s\") pod \"payload-processing-e2e-shared-b-w3-d844cc-6c8446bb85-frh5c\" (UID: \"fa552843-e4cf-447a-a25d-288eb7f9660a\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-d844cc-6c8446bb85-frh5c"
Sep 11 12:07:51.762594 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:51.762555    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-d844cc-6c8446bb85-frh5c"
Sep 11 12:07:52.042197 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:52.042167    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-d844cc-6c8446bb85-frh5c"]
Sep 11 12:07:52.043710 ip-10-0-142-145 kubenswrapper[2579]: W0911 12:07:52.043681    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa552843_e4cf_447a_a25d_288eb7f9660a.slice/crio-a991b0e0427dc502e367aaa4c06d0c8802214b07184d8217297660f391f7bfd9 WatchSource:0}: Error finding container a991b0e0427dc502e367aaa4c06d0c8802214b07184d8217297660f391f7bfd9: Status 404 returned error can't find the container with id a991b0e0427dc502e367aaa4c06d0c8802214b07184d8217297660f391f7bfd9
Sep 11 12:07:52.450448 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:52.450409    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-d844cc-6c8446bb85-frh5c" event={"ID":"fa552843-e4cf-447a-a25d-288eb7f9660a","Type":"ContainerStarted","Data":"999122a4e67477dfc122d9223f0f9c59e4edf90de5f1bd16001644bdb963768e"}
Sep 11 12:07:52.450676 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:52.450478    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-d844cc-6c8446bb85-frh5c" event={"ID":"fa552843-e4cf-447a-a25d-288eb7f9660a","Type":"ContainerStarted","Data":"a991b0e0427dc502e367aaa4c06d0c8802214b07184d8217297660f391f7bfd9"}
Sep 11 12:07:52.450676 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:52.450541    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-d844cc-6c8446bb85-frh5c"
Sep 11 12:07:52.471200 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:52.471154    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-d844cc-6c8446bb85-frh5c" podStartSLOduration=1.471137628 podStartE2EDuration="1.471137628s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:07:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:07:52.46946936 +0000 UTC m=+1176.366911107" watchObservedRunningTime="2026-09-11 12:07:52.471137628 +0000 UTC m=+1176.368579373"
Sep 11 12:07:53.874007 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:53.873974    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:07:56.167341 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:56.167308    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:56.275665 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:56.275627    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:57.052836 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:57.052801    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:57.352728 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:57.352697    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6"]
Sep 11 12:07:57.353228 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:57.353212    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="16aaa4c7-eb22-436e-ab60-12d4c78a732e" containerName="istio-proxy"
Sep 11 12:07:57.356053 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:57.356033    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6"
Sep 11 12:07:57.358708 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:57.358684    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-45352b-openshift-default-dockercfg-9wptp\""
Sep 11 12:07:57.374639 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:57.374612    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6"]
Sep 11 12:07:57.389324 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:57.389263    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:57.415535 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:57.415506    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/650ba6c3-36f7-4f49-9cf7-230d2dcc7d77-istiod-ca-cert\") pod \"e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6\" (UID: \"650ba6c3-36f7-4f49-9cf7-230d2dcc7d77\") " pod="openshift-ingress/e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6"
Sep 11 12:07:57.415683 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:57.415552    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/650ba6c3-36f7-4f49-9cf7-230d2dcc7d77-workload-certs\") pod \"e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6\" (UID: \"650ba6c3-36f7-4f49-9cf7-230d2dcc7d77\") " pod="openshift-ingress/e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6"
Sep 11 12:07:57.415683 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:57.415573    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/650ba6c3-36f7-4f49-9cf7-230d2dcc7d77-istio-token\") pod \"e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6\" (UID: \"650ba6c3-36f7-4f49-9cf7-230d2dcc7d77\") " pod="openshift-ingress/e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6"
Sep 11 12:07:57.415683 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:57.415620    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/650ba6c3-36f7-4f49-9cf7-230d2dcc7d77-workload-socket\") pod \"e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6\" (UID: \"650ba6c3-36f7-4f49-9cf7-230d2dcc7d77\") " pod="openshift-ingress/e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6"
Sep 11 12:07:57.415683 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:57.415671    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/650ba6c3-36f7-4f49-9cf7-230d2dcc7d77-istio-data\") pod \"e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6\" (UID: \"650ba6c3-36f7-4f49-9cf7-230d2dcc7d77\") " pod="openshift-ingress/e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6"
Sep 11 12:07:57.415835 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:57.415723    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/650ba6c3-36f7-4f49-9cf7-230d2dcc7d77-credential-socket\") pod \"e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6\" (UID: \"650ba6c3-36f7-4f49-9cf7-230d2dcc7d77\") " pod="openshift-ingress/e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6"
Sep 11 12:07:57.415835 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:57.415754    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flp7t\" (UniqueName: \"kubernetes.io/projected/650ba6c3-36f7-4f49-9cf7-230d2dcc7d77-kube-api-access-flp7t\") pod \"e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6\" (UID: \"650ba6c3-36f7-4f49-9cf7-230d2dcc7d77\") " pod="openshift-ingress/e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6"
Sep 11 12:07:57.415908 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:57.415836    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/650ba6c3-36f7-4f49-9cf7-230d2dcc7d77-istio-podinfo\") pod \"e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6\" (UID: \"650ba6c3-36f7-4f49-9cf7-230d2dcc7d77\") " pod="openshift-ingress/e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6"
Sep 11 12:07:57.415908 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:57.415864    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/650ba6c3-36f7-4f49-9cf7-230d2dcc7d77-istio-envoy\") pod \"e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6\" (UID: \"650ba6c3-36f7-4f49-9cf7-230d2dcc7d77\") " pod="openshift-ingress/e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6"
Sep 11 12:07:57.516970 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:57.516927    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/650ba6c3-36f7-4f49-9cf7-230d2dcc7d77-workload-socket\") pod \"e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6\" (UID: \"650ba6c3-36f7-4f49-9cf7-230d2dcc7d77\") " pod="openshift-ingress/e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6"
Sep 11 12:07:57.516970 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:57.516979    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/650ba6c3-36f7-4f49-9cf7-230d2dcc7d77-istio-data\") pod \"e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6\" (UID: \"650ba6c3-36f7-4f49-9cf7-230d2dcc7d77\") " pod="openshift-ingress/e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6"
Sep 11 12:07:57.517247 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:57.517004    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/650ba6c3-36f7-4f49-9cf7-230d2dcc7d77-credential-socket\") pod \"e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6\" (UID: \"650ba6c3-36f7-4f49-9cf7-230d2dcc7d77\") " pod="openshift-ingress/e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6"
Sep 11 12:07:57.517247 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:57.517019    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flp7t\" (UniqueName: \"kubernetes.io/projected/650ba6c3-36f7-4f49-9cf7-230d2dcc7d77-kube-api-access-flp7t\") pod \"e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6\" (UID: \"650ba6c3-36f7-4f49-9cf7-230d2dcc7d77\") " pod="openshift-ingress/e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6"
Sep 11 12:07:57.517247 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:57.517060    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/650ba6c3-36f7-4f49-9cf7-230d2dcc7d77-istio-podinfo\") pod \"e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6\" (UID: \"650ba6c3-36f7-4f49-9cf7-230d2dcc7d77\") " pod="openshift-ingress/e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6"
Sep 11 12:07:57.517247 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:57.517082    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/650ba6c3-36f7-4f49-9cf7-230d2dcc7d77-istio-envoy\") pod \"e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6\" (UID: \"650ba6c3-36f7-4f49-9cf7-230d2dcc7d77\") " pod="openshift-ingress/e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6"
Sep 11 12:07:57.517247 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:57.517108    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/650ba6c3-36f7-4f49-9cf7-230d2dcc7d77-istiod-ca-cert\") pod \"e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6\" (UID: \"650ba6c3-36f7-4f49-9cf7-230d2dcc7d77\") " pod="openshift-ingress/e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6"
Sep 11 12:07:57.517247 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:57.517143    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/650ba6c3-36f7-4f49-9cf7-230d2dcc7d77-workload-certs\") pod \"e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6\" (UID: \"650ba6c3-36f7-4f49-9cf7-230d2dcc7d77\") " pod="openshift-ingress/e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6"
Sep 11 12:07:57.517247 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:57.517162    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/650ba6c3-36f7-4f49-9cf7-230d2dcc7d77-istio-token\") pod \"e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6\" (UID: \"650ba6c3-36f7-4f49-9cf7-230d2dcc7d77\") " pod="openshift-ingress/e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6"
Sep 11 12:07:57.517655 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:57.517431    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/650ba6c3-36f7-4f49-9cf7-230d2dcc7d77-workload-socket\") pod \"e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6\" (UID: \"650ba6c3-36f7-4f49-9cf7-230d2dcc7d77\") " pod="openshift-ingress/e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6"
Sep 11 12:07:57.517714 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:57.517663    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/650ba6c3-36f7-4f49-9cf7-230d2dcc7d77-istio-data\") pod \"e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6\" (UID: \"650ba6c3-36f7-4f49-9cf7-230d2dcc7d77\") " pod="openshift-ingress/e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6"
Sep 11 12:07:57.518120 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:57.518092    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/650ba6c3-36f7-4f49-9cf7-230d2dcc7d77-credential-socket\") pod \"e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6\" (UID: \"650ba6c3-36f7-4f49-9cf7-230d2dcc7d77\") " pod="openshift-ingress/e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6"
Sep 11 12:07:57.518237 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:57.518118    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/650ba6c3-36f7-4f49-9cf7-230d2dcc7d77-workload-certs\") pod \"e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6\" (UID: \"650ba6c3-36f7-4f49-9cf7-230d2dcc7d77\") " pod="openshift-ingress/e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6"
Sep 11 12:07:57.518449 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:57.518429    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/650ba6c3-36f7-4f49-9cf7-230d2dcc7d77-istiod-ca-cert\") pod \"e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6\" (UID: \"650ba6c3-36f7-4f49-9cf7-230d2dcc7d77\") " pod="openshift-ingress/e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6"
Sep 11 12:07:57.519707 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:57.519682    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/650ba6c3-36f7-4f49-9cf7-230d2dcc7d77-istio-envoy\") pod \"e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6\" (UID: \"650ba6c3-36f7-4f49-9cf7-230d2dcc7d77\") " pod="openshift-ingress/e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6"
Sep 11 12:07:57.519919 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:57.519902    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/650ba6c3-36f7-4f49-9cf7-230d2dcc7d77-istio-podinfo\") pod \"e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6\" (UID: \"650ba6c3-36f7-4f49-9cf7-230d2dcc7d77\") " pod="openshift-ingress/e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6"
Sep 11 12:07:57.527370 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:57.527340    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/650ba6c3-36f7-4f49-9cf7-230d2dcc7d77-istio-token\") pod \"e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6\" (UID: \"650ba6c3-36f7-4f49-9cf7-230d2dcc7d77\") " pod="openshift-ingress/e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6"
Sep 11 12:07:57.527537 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:57.527512    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-flp7t\" (UniqueName: \"kubernetes.io/projected/650ba6c3-36f7-4f49-9cf7-230d2dcc7d77-kube-api-access-flp7t\") pod \"e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6\" (UID: \"650ba6c3-36f7-4f49-9cf7-230d2dcc7d77\") " pod="openshift-ingress/e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6"
Sep 11 12:07:57.666622 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:57.666531    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6"
Sep 11 12:07:57.801510 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:57.799844    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:57.963442 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:57.963413    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6"]
Sep 11 12:07:57.965899 ip-10-0-142-145 kubenswrapper[2579]: W0911 12:07:57.965869    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod650ba6c3_36f7_4f49_9cf7_230d2dcc7d77.slice/crio-bf512af858cbf586a5738eb610a96597957b43f9003ddc7c20c45083192f34be WatchSource:0}: Error finding container bf512af858cbf586a5738eb610a96597957b43f9003ddc7c20c45083192f34be: Status 404 returned error can't find the container with id bf512af858cbf586a5738eb610a96597957b43f9003ddc7c20c45083192f34be
Sep 11 12:07:57.968266 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:57.968232    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 12:07:57.968371 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:57.968314    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 12:07:57.968371 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:57.968359    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 12:07:58.503417 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:58.503008    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6" event={"ID":"650ba6c3-36f7-4f49-9cf7-230d2dcc7d77","Type":"ContainerStarted","Data":"fd31c201c55293320d1f67a85243080b52b5a4c08bf2561f706257131f72c2f3"}
Sep 11 12:07:58.503417 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:58.503097    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6" event={"ID":"650ba6c3-36f7-4f49-9cf7-230d2dcc7d77","Type":"ContainerStarted","Data":"bf512af858cbf586a5738eb610a96597957b43f9003ddc7c20c45083192f34be"}
Sep 11 12:07:58.530841 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:58.530783    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6" podStartSLOduration=1.530765157 podStartE2EDuration="1.530765157s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:07:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:07:58.529374822 +0000 UTC m=+1182.426816568" watchObservedRunningTime="2026-09-11 12:07:58.530765157 +0000 UTC m=+1182.428206901"
Sep 11 12:07:58.667569 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:58.667533    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6"
Sep 11 12:07:59.671772 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:07:59.671742    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6"
Sep 11 12:08:00.486697 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:00.486655    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:00.514626 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:00.514597    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6"
Sep 11 12:08:00.515685 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:00.515658    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6"
Sep 11 12:08:00.589272 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:00.589242    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:01.703884 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:01.703837    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:01.807768 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:01.807728    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:02.318118 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:02.318077    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-45352b-5ddc97f6c8-tv8dc"]
Sep 11 12:08:02.321589 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:02.321564    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-45352b-5ddc97f6c8-tv8dc"
Sep 11 12:08:02.324202 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:02.324177    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-45352b\""
Sep 11 12:08:02.324358 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:02.324180    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-45352b-dockercfg-xxx7x\""
Sep 11 12:08:02.334726 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:02.334437    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:02.335470 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:02.335403    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-45352b-5ddc97f6c8-tv8dc"]
Sep 11 12:08:02.469611 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:02.469576    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ef64f99-964e-4117-b678-2d0f7d025d66-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-45352b-5ddc97f6c8-tv8dc\" (UID: \"7ef64f99-964e-4117-b678-2d0f7d025d66\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-45352b-5ddc97f6c8-tv8dc"
Sep 11 12:08:02.469786 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:02.469629    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5h6b7\" (UniqueName: \"kubernetes.io/projected/7ef64f99-964e-4117-b678-2d0f7d025d66-kube-api-access-5h6b7\") pod \"payload-pre-processing-e2e-shared-a-w2-45352b-5ddc97f6c8-tv8dc\" (UID: \"7ef64f99-964e-4117-b678-2d0f7d025d66\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-45352b-5ddc97f6c8-tv8dc"
Sep 11 12:08:02.480045 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:02.480000    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:02.570762 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:02.570656    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ef64f99-964e-4117-b678-2d0f7d025d66-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-45352b-5ddc97f6c8-tv8dc\" (UID: \"7ef64f99-964e-4117-b678-2d0f7d025d66\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-45352b-5ddc97f6c8-tv8dc"
Sep 11 12:08:02.570762 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:02.570740    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5h6b7\" (UniqueName: \"kubernetes.io/projected/7ef64f99-964e-4117-b678-2d0f7d025d66-kube-api-access-5h6b7\") pod \"payload-pre-processing-e2e-shared-a-w2-45352b-5ddc97f6c8-tv8dc\" (UID: \"7ef64f99-964e-4117-b678-2d0f7d025d66\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-45352b-5ddc97f6c8-tv8dc"
Sep 11 12:08:02.571365 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:02.571337    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ef64f99-964e-4117-b678-2d0f7d025d66-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-45352b-5ddc97f6c8-tv8dc\" (UID: \"7ef64f99-964e-4117-b678-2d0f7d025d66\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-45352b-5ddc97f6c8-tv8dc"
Sep 11 12:08:02.581028 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:02.580993    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5h6b7\" (UniqueName: \"kubernetes.io/projected/7ef64f99-964e-4117-b678-2d0f7d025d66-kube-api-access-5h6b7\") pod \"payload-pre-processing-e2e-shared-a-w2-45352b-5ddc97f6c8-tv8dc\" (UID: \"7ef64f99-964e-4117-b678-2d0f7d025d66\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-45352b-5ddc97f6c8-tv8dc"
Sep 11 12:08:02.585024 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:02.583612    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:02.634042 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:02.633996    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-45352b-5ddc97f6c8-tv8dc"
Sep 11 12:08:02.911589 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:02.911559    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-45352b-5ddc97f6c8-tv8dc"]
Sep 11 12:08:02.913348 ip-10-0-142-145 kubenswrapper[2579]: W0911 12:08:02.913319    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ef64f99_964e_4117_b678_2d0f7d025d66.slice/crio-a053f80a656996b13f7bdef87d38aa96d56f90efd0f0a208f8c09a08693a529a WatchSource:0}: Error finding container a053f80a656996b13f7bdef87d38aa96d56f90efd0f0a208f8c09a08693a529a: Status 404 returned error can't find the container with id a053f80a656996b13f7bdef87d38aa96d56f90efd0f0a208f8c09a08693a529a
Sep 11 12:08:03.457981 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:03.457949    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-d844cc-6c8446bb85-frh5c"
Sep 11 12:08:03.533034 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:03.532996    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-45352b-5ddc97f6c8-tv8dc" event={"ID":"7ef64f99-964e-4117-b678-2d0f7d025d66","Type":"ContainerStarted","Data":"43251685ac897dd4481322cefcf6d68ae56133c499021539dbe8aed9a23542e1"}
Sep 11 12:08:03.533189 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:03.533064    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-45352b-5ddc97f6c8-tv8dc" event={"ID":"7ef64f99-964e-4117-b678-2d0f7d025d66","Type":"ContainerStarted","Data":"a053f80a656996b13f7bdef87d38aa96d56f90efd0f0a208f8c09a08693a529a"}
Sep 11 12:08:03.533254 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:03.533190    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-45352b-5ddc97f6c8-tv8dc"
Sep 11 12:08:03.551899 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:03.551843    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-45352b-5ddc97f6c8-tv8dc" podStartSLOduration=1.55182599 podStartE2EDuration="1.55182599s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:08:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:08:03.549857691 +0000 UTC m=+1187.447299438" watchObservedRunningTime="2026-09-11 12:08:03.55182599 +0000 UTC m=+1187.449267735"
Sep 11 12:08:04.117992 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:04.117678    2579 patch_prober.go:28] interesting pod/e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": read tcp 10.134.0.2:57774->10.134.0.56:15021: read: connection reset by peer" start-of-body=
Sep 11 12:08:04.117992 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:04.117761    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65" podUID="45e86e88-5ccd-4c4c-8d31-8350e844f996" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": read tcp 10.134.0.2:57774->10.134.0.56:15021: read: connection reset by peer"
Sep 11 12:08:04.541473 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:04.541364    2579 generic.go:356] "Generic (PLEG): container finished" podID="45e86e88-5ccd-4c4c-8d31-8350e844f996" containerID="8ddbae16db110bb06d24ae0afb63573f49c1581c4f9550ba5ccd2292f86c5c2c" exitCode=0
Sep 11 12:08:04.541473 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:04.541440    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65" event={"ID":"45e86e88-5ccd-4c4c-8d31-8350e844f996","Type":"ContainerDied","Data":"8ddbae16db110bb06d24ae0afb63573f49c1581c4f9550ba5ccd2292f86c5c2c"}
Sep 11 12:08:04.541900 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:04.541880    2579 scope.go:117] "RemoveContainer" containerID="8ddbae16db110bb06d24ae0afb63573f49c1581c4f9550ba5ccd2292f86c5c2c"
Sep 11 12:08:04.542997 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:04.542972    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 12:08:04.543059 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:04.543040    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 12:08:04.543108 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:04.543072    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 12:08:05.549736 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:05.549702    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65" event={"ID":"45e86e88-5ccd-4c4c-8d31-8350e844f996","Type":"ContainerStarted","Data":"137cee4919336a89de681a2d292b81ce720e3d0fe244a7d20ec740435414c952"}
Sep 11 12:08:06.064182 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:06.064146    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:08:06.508065 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:06.508023    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65"
Sep 11 12:08:06.508304 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:06.508276    2579 patch_prober.go:28] interesting pod/e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Sep 11 12:08:06.508424 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:06.508336    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65" podUID="45e86e88-5ccd-4c4c-8d31-8350e844f996" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Sep 11 12:08:07.508709 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:07.508662    2579 patch_prober.go:28] interesting pod/e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Sep 11 12:08:07.509116 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:07.508744    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65" podUID="45e86e88-5ccd-4c4c-8d31-8350e844f996" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Sep 11 12:08:08.508828 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:08.508777    2579 patch_prober.go:28] interesting pod/e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Sep 11 12:08:08.509341 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:08.508856    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65" podUID="45e86e88-5ccd-4c4c-8d31-8350e844f996" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Sep 11 12:08:09.508587 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:09.508544    2579 patch_prober.go:28] interesting pod/e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Sep 11 12:08:09.508799 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:09.508612    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65" podUID="45e86e88-5ccd-4c4c-8d31-8350e844f996" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Sep 11 12:08:10.507792 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:10.507756    2579 patch_prober.go:28] interesting pod/e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Sep 11 12:08:10.508212 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:10.507817    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65" podUID="45e86e88-5ccd-4c4c-8d31-8350e844f996" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Sep 11 12:08:11.511344 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:11.511313    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65"
Sep 11 12:08:11.511779 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:11.511589    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65"
Sep 11 12:08:11.512276 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:11.512260    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65"
Sep 11 12:08:11.756511 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:11.756465    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:14.362048 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:14.362010    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:14.544261 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:14.544228    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-45352b-5ddc97f6c8-tv8dc"
Sep 11 12:08:16.369110 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:16.368981    2579 patch_prober.go:28] interesting pod/e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": read tcp 10.134.0.2:59672->10.134.0.58:15021: read: connection reset by peer" start-of-body=
Sep 11 12:08:16.369110 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:16.369057    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6" podUID="650ba6c3-36f7-4f49-9cf7-230d2dcc7d77" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": read tcp 10.134.0.2:59672->10.134.0.58:15021: read: connection reset by peer"
Sep 11 12:08:16.552359 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:16.552279    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-ch96n_1f0f9fa5-cbd1-468b-ae81-82b48c101d37/console-operator/2.log"
Sep 11 12:08:16.554016 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:16.552940    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-ch96n_1f0f9fa5-cbd1-468b-ae81-82b48c101d37/console-operator/2.log"
Sep 11 12:08:16.628493 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:16.628374    2579 generic.go:356] "Generic (PLEG): container finished" podID="650ba6c3-36f7-4f49-9cf7-230d2dcc7d77" containerID="fd31c201c55293320d1f67a85243080b52b5a4c08bf2561f706257131f72c2f3" exitCode=0
Sep 11 12:08:16.628493 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:16.628446    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6" event={"ID":"650ba6c3-36f7-4f49-9cf7-230d2dcc7d77","Type":"ContainerDied","Data":"fd31c201c55293320d1f67a85243080b52b5a4c08bf2561f706257131f72c2f3"}
Sep 11 12:08:16.628914 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:16.628890    2579 scope.go:117] "RemoveContainer" containerID="fd31c201c55293320d1f67a85243080b52b5a4c08bf2561f706257131f72c2f3"
Sep 11 12:08:16.630011 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:16.629981    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 12:08:16.630067 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:16.630052    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 12:08:16.630104 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:16.630091    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 12:08:17.636178 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:17.636143    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6" event={"ID":"650ba6c3-36f7-4f49-9cf7-230d2dcc7d77","Type":"ContainerStarted","Data":"5a6e84002e9a77ca84bf788bf6be258f970f9d55c83f8a8f22de04fa2a1555e4"}
Sep 11 12:08:17.667637 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:17.667604    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6"
Sep 11 12:08:17.667829 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:17.667799    2579 patch_prober.go:28] interesting pod/e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Sep 11 12:08:17.667892 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:17.667834    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6" podUID="650ba6c3-36f7-4f49-9cf7-230d2dcc7d77" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Sep 11 12:08:18.667052 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:18.667016    2579 patch_prober.go:28] interesting pod/e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Sep 11 12:08:18.667513 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:18.667071    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6" podUID="650ba6c3-36f7-4f49-9cf7-230d2dcc7d77" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Sep 11 12:08:19.667060 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:19.667028    2579 patch_prober.go:28] interesting pod/e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Sep 11 12:08:19.667535 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:19.667078    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6" podUID="650ba6c3-36f7-4f49-9cf7-230d2dcc7d77" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Sep 11 12:08:20.667190 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:20.667154    2579 patch_prober.go:28] interesting pod/e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Sep 11 12:08:20.667686 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:20.667213    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6" podUID="650ba6c3-36f7-4f49-9cf7-230d2dcc7d77" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Sep 11 12:08:21.666990 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:21.666955    2579 patch_prober.go:28] interesting pod/e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Sep 11 12:08:21.667170 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:21.667011    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6" podUID="650ba6c3-36f7-4f49-9cf7-230d2dcc7d77" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Sep 11 12:08:22.617308 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:22.617254    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:22.667225 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:22.667191    2579 patch_prober.go:28] interesting pod/e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Sep 11 12:08:22.667406 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:22.667248    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6" podUID="650ba6c3-36f7-4f49-9cf7-230d2dcc7d77" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Sep 11 12:08:23.671616 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:23.671589    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6"
Sep 11 12:08:23.672017 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:23.671968    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6"
Sep 11 12:08:23.672673 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:23.672648    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6"
Sep 11 12:08:25.415749 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:25.415713    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:25.739819 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:25.739730    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:26.345944 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:26.345903    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:26.445459 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:26.445414    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:29.035161 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:29.035125    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:29.160754 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:29.160715    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:30.181848 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:30.181814    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:30.284724 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:30.284684    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:30.382423 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:30.382376    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:30.737584 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:30.737534    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:31.020772 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:31.020686    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:31.047209 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:31.047176    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-f7851b-576c644c89-nbcgn"]
Sep 11 12:08:31.050711 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:31.050694    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f7851b-576c644c89-nbcgn"
Sep 11 12:08:31.055798 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:31.055776    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-f7851b\""
Sep 11 12:08:31.057787 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:31.057766    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-f7851b-dockercfg-xxw4b\""
Sep 11 12:08:31.070039 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:31.070010    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-f7851b-576c644c89-nbcgn"]
Sep 11 12:08:31.152047 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:31.152005    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5722f04-1a5f-4781-b946-5972261d9a9c-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-f7851b-576c644c89-nbcgn\" (UID: \"b5722f04-1a5f-4781-b946-5972261d9a9c\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f7851b-576c644c89-nbcgn"
Sep 11 12:08:31.152239 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:31.152167    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7844s\" (UniqueName: \"kubernetes.io/projected/b5722f04-1a5f-4781-b946-5972261d9a9c-kube-api-access-7844s\") pod \"payload-processing-e2e-shared-b-w2-f7851b-576c644c89-nbcgn\" (UID: \"b5722f04-1a5f-4781-b946-5972261d9a9c\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f7851b-576c644c89-nbcgn"
Sep 11 12:08:31.253077 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:31.253035    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7844s\" (UniqueName: \"kubernetes.io/projected/b5722f04-1a5f-4781-b946-5972261d9a9c-kube-api-access-7844s\") pod \"payload-processing-e2e-shared-b-w2-f7851b-576c644c89-nbcgn\" (UID: \"b5722f04-1a5f-4781-b946-5972261d9a9c\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f7851b-576c644c89-nbcgn"
Sep 11 12:08:31.253572 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:31.253146    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5722f04-1a5f-4781-b946-5972261d9a9c-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-f7851b-576c644c89-nbcgn\" (UID: \"b5722f04-1a5f-4781-b946-5972261d9a9c\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f7851b-576c644c89-nbcgn"
Sep 11 12:08:31.253860 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:31.253840    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5722f04-1a5f-4781-b946-5972261d9a9c-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-f7851b-576c644c89-nbcgn\" (UID: \"b5722f04-1a5f-4781-b946-5972261d9a9c\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f7851b-576c644c89-nbcgn"
Sep 11 12:08:31.262346 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:31.262323    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7844s\" (UniqueName: \"kubernetes.io/projected/b5722f04-1a5f-4781-b946-5972261d9a9c-kube-api-access-7844s\") pod \"payload-processing-e2e-shared-b-w2-f7851b-576c644c89-nbcgn\" (UID: \"b5722f04-1a5f-4781-b946-5972261d9a9c\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f7851b-576c644c89-nbcgn"
Sep 11 12:08:31.360684 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:31.360642    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f7851b-576c644c89-nbcgn"
Sep 11 12:08:31.634230 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:31.634204    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-f7851b-576c644c89-nbcgn"]
Sep 11 12:08:31.637222 ip-10-0-142-145 kubenswrapper[2579]: W0911 12:08:31.637185    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5722f04_1a5f_4781_b946_5972261d9a9c.slice/crio-af08e6e6e05712fc8ca5cf1f5074e6cd326e094bf77bbefa6ed28540fe1f8047 WatchSource:0}: Error finding container af08e6e6e05712fc8ca5cf1f5074e6cd326e094bf77bbefa6ed28540fe1f8047: Status 404 returned error can't find the container with id af08e6e6e05712fc8ca5cf1f5074e6cd326e094bf77bbefa6ed28540fe1f8047
Sep 11 12:08:31.713539 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:31.713506    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f7851b-576c644c89-nbcgn" event={"ID":"b5722f04-1a5f-4781-b946-5972261d9a9c","Type":"ContainerStarted","Data":"9a048ed8f208402a25f7c2ab71c02da167da2fda24306f0d0502feba8b4722a9"}
Sep 11 12:08:31.713714 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:31.713564    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f7851b-576c644c89-nbcgn" event={"ID":"b5722f04-1a5f-4781-b946-5972261d9a9c","Type":"ContainerStarted","Data":"af08e6e6e05712fc8ca5cf1f5074e6cd326e094bf77bbefa6ed28540fe1f8047"}
Sep 11 12:08:31.713714 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:31.713669    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f7851b-576c644c89-nbcgn"
Sep 11 12:08:31.740186 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:31.740128    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f7851b-576c644c89-nbcgn" podStartSLOduration=0.740109267 podStartE2EDuration="740.109267ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:08:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:08:31.738883749 +0000 UTC m=+1215.636325496" watchObservedRunningTime="2026-09-11 12:08:31.740109267 +0000 UTC m=+1215.637551011"
Sep 11 12:08:32.226643 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:32.226606    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:36.875189 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:36.875153    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:08:42.722332 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:42.722301    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f7851b-576c644c89-nbcgn"
Sep 11 12:08:53.956083 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:08:53.956037    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:02.770585 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:09:02.770540    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:04.469402 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:09:04.469353    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:04.755727 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:09:04.755635    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:04.899994 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:09:04.899960    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:08.248988 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:09:08.248923    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:08.328791 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:09:08.328755    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:08.798781 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:09:08.798695    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:09:09.528973 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:09:09.528940    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8d375e02-87b587458-7m5sf"]
Sep 11 12:09:09.532126 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:09:09.532107    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8d375e02-87b587458-7m5sf"
Sep 11 12:09:09.536352 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:09:09.536321    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 11 12:09:09.536352 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:09:09.536340    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-8d375e02-serving-cert\""
Sep 11 12:09:09.536571 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:09:09.536379    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 11 12:09:09.536571 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:09:09.536347    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-rrxjh\""
Sep 11 12:09:09.536571 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:09:09.536325    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 12:09:09.548035 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:09:09.547983    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8d375e02-87b587458-7m5sf"]
Sep 11 12:09:09.574621 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:09:09.574589    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:09.621318 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:09:09.621278    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7h9wl\" (UniqueName: \"kubernetes.io/projected/22bf031b-a2cd-4d6e-b24f-5064612ee47a-kube-api-access-7h9wl\") pod \"maas-api-e2e-mt-8d375e02-87b587458-7m5sf\" (UID: \"22bf031b-a2cd-4d6e-b24f-5064612ee47a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8d375e02-87b587458-7m5sf"
Sep 11 12:09:09.621533 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:09:09.621338    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/22bf031b-a2cd-4d6e-b24f-5064612ee47a-openshift-service-ca\") pod \"maas-api-e2e-mt-8d375e02-87b587458-7m5sf\" (UID: \"22bf031b-a2cd-4d6e-b24f-5064612ee47a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8d375e02-87b587458-7m5sf"
Sep 11 12:09:09.621533 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:09:09.621423    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/22bf031b-a2cd-4d6e-b24f-5064612ee47a-maas-api-tls\") pod \"maas-api-e2e-mt-8d375e02-87b587458-7m5sf\" (UID: \"22bf031b-a2cd-4d6e-b24f-5064612ee47a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8d375e02-87b587458-7m5sf"
Sep 11 12:09:09.621533 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:09:09.621480    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/22bf031b-a2cd-4d6e-b24f-5064612ee47a-metrics-certs\") pod \"maas-api-e2e-mt-8d375e02-87b587458-7m5sf\" (UID: \"22bf031b-a2cd-4d6e-b24f-5064612ee47a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8d375e02-87b587458-7m5sf"
Sep 11 12:09:09.671652 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:09:09.671622    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:09.722494 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:09:09.722462    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/22bf031b-a2cd-4d6e-b24f-5064612ee47a-metrics-certs\") pod \"maas-api-e2e-mt-8d375e02-87b587458-7m5sf\" (UID: \"22bf031b-a2cd-4d6e-b24f-5064612ee47a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8d375e02-87b587458-7m5sf"
Sep 11 12:09:09.722705 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:09:09.722525    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7h9wl\" (UniqueName: \"kubernetes.io/projected/22bf031b-a2cd-4d6e-b24f-5064612ee47a-kube-api-access-7h9wl\") pod \"maas-api-e2e-mt-8d375e02-87b587458-7m5sf\" (UID: \"22bf031b-a2cd-4d6e-b24f-5064612ee47a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8d375e02-87b587458-7m5sf"
Sep 11 12:09:09.722705 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:09:09.722556    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/22bf031b-a2cd-4d6e-b24f-5064612ee47a-openshift-service-ca\") pod \"maas-api-e2e-mt-8d375e02-87b587458-7m5sf\" (UID: \"22bf031b-a2cd-4d6e-b24f-5064612ee47a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8d375e02-87b587458-7m5sf"
Sep 11 12:09:09.722705 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:09:09.722600    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/22bf031b-a2cd-4d6e-b24f-5064612ee47a-maas-api-tls\") pod \"maas-api-e2e-mt-8d375e02-87b587458-7m5sf\" (UID: \"22bf031b-a2cd-4d6e-b24f-5064612ee47a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8d375e02-87b587458-7m5sf"
Sep 11 12:09:09.723320 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:09:09.723289    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/22bf031b-a2cd-4d6e-b24f-5064612ee47a-openshift-service-ca\") pod \"maas-api-e2e-mt-8d375e02-87b587458-7m5sf\" (UID: \"22bf031b-a2cd-4d6e-b24f-5064612ee47a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8d375e02-87b587458-7m5sf"
Sep 11 12:09:09.724989 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:09:09.724965    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/22bf031b-a2cd-4d6e-b24f-5064612ee47a-metrics-certs\") pod \"maas-api-e2e-mt-8d375e02-87b587458-7m5sf\" (UID: \"22bf031b-a2cd-4d6e-b24f-5064612ee47a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8d375e02-87b587458-7m5sf"
Sep 11 12:09:09.725154 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:09:09.725136    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/22bf031b-a2cd-4d6e-b24f-5064612ee47a-maas-api-tls\") pod \"maas-api-e2e-mt-8d375e02-87b587458-7m5sf\" (UID: \"22bf031b-a2cd-4d6e-b24f-5064612ee47a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8d375e02-87b587458-7m5sf"
Sep 11 12:09:09.731716 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:09:09.731692    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7h9wl\" (UniqueName: \"kubernetes.io/projected/22bf031b-a2cd-4d6e-b24f-5064612ee47a-kube-api-access-7h9wl\") pod \"maas-api-e2e-mt-8d375e02-87b587458-7m5sf\" (UID: \"22bf031b-a2cd-4d6e-b24f-5064612ee47a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8d375e02-87b587458-7m5sf"
Sep 11 12:09:09.774263 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:09:09.774226    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:09.844670 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:09:09.844638    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8d375e02-87b587458-7m5sf"
Sep 11 12:09:10.131575 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:09:10.131547    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8d375e02-87b587458-7m5sf"]
Sep 11 12:09:10.134295 ip-10-0-142-145 kubenswrapper[2579]: W0911 12:09:10.134264    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22bf031b_a2cd_4d6e_b24f_5064612ee47a.slice/crio-9af53fdac25dcf6ca1785ffed826179c35218d256f39f13d9154f140aaeba099 WatchSource:0}: Error finding container 9af53fdac25dcf6ca1785ffed826179c35218d256f39f13d9154f140aaeba099: Status 404 returned error can't find the container with id 9af53fdac25dcf6ca1785ffed826179c35218d256f39f13d9154f140aaeba099
Sep 11 12:09:10.136039 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:09:10.136015    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 12:09:10.184010 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:09:10.183979    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:10.297586 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:09:10.297549    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-8d375e02-99f8c946-tvcq6"]
Sep 11 12:09:10.300946 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:09:10.300929    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-8d375e02-99f8c946-tvcq6"
Sep 11 12:09:10.303581 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:09:10.303552    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-8d375e02\""
Sep 11 12:09:10.303714 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:09:10.303612    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-8d375e02-dockercfg-p7sf2\""
Sep 11 12:09:10.310918 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:09:10.310894    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-8d375e02-99f8c946-tvcq6"]
Sep 11 12:09:10.327509 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:09:10.327474    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab9a95e3-2546-482a-a9cb-b11500fd901c-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-8d375e02-99f8c946-tvcq6\" (UID: \"ab9a95e3-2546-482a-a9cb-b11500fd901c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-8d375e02-99f8c946-tvcq6"
Sep 11 12:09:10.327693 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:09:10.327534    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbp4f\" (UniqueName: \"kubernetes.io/projected/ab9a95e3-2546-482a-a9cb-b11500fd901c-kube-api-access-mbp4f\") pod \"payload-pre-processing-e2e-mt-8d375e02-99f8c946-tvcq6\" (UID: \"ab9a95e3-2546-482a-a9cb-b11500fd901c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-8d375e02-99f8c946-tvcq6"
Sep 11 12:09:10.428740 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:09:10.428649    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab9a95e3-2546-482a-a9cb-b11500fd901c-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-8d375e02-99f8c946-tvcq6\" (UID: \"ab9a95e3-2546-482a-a9cb-b11500fd901c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-8d375e02-99f8c946-tvcq6"
Sep 11 12:09:10.428903 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:09:10.428780    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbp4f\" (UniqueName: \"kubernetes.io/projected/ab9a95e3-2546-482a-a9cb-b11500fd901c-kube-api-access-mbp4f\") pod \"payload-pre-processing-e2e-mt-8d375e02-99f8c946-tvcq6\" (UID: \"ab9a95e3-2546-482a-a9cb-b11500fd901c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-8d375e02-99f8c946-tvcq6"
Sep 11 12:09:10.429434 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:09:10.429366    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab9a95e3-2546-482a-a9cb-b11500fd901c-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-8d375e02-99f8c946-tvcq6\" (UID: \"ab9a95e3-2546-482a-a9cb-b11500fd901c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-8d375e02-99f8c946-tvcq6"
Sep 11 12:09:10.438925 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:09:10.438860    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbp4f\" (UniqueName: \"kubernetes.io/projected/ab9a95e3-2546-482a-a9cb-b11500fd901c-kube-api-access-mbp4f\") pod \"payload-pre-processing-e2e-mt-8d375e02-99f8c946-tvcq6\" (UID: \"ab9a95e3-2546-482a-a9cb-b11500fd901c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-8d375e02-99f8c946-tvcq6"
Sep 11 12:09:10.554581 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:09:10.554543    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:10.613356 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:09:10.613327    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-8d375e02-99f8c946-tvcq6"
Sep 11 12:09:10.884654 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:09:10.884629    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-8d375e02-99f8c946-tvcq6"]
Sep 11 12:09:10.887096 ip-10-0-142-145 kubenswrapper[2579]: W0911 12:09:10.887068    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab9a95e3_2546_482a_a9cb_b11500fd901c.slice/crio-1f459a9b667afdfb7d8e50d196ff14a570d1c40a8ba5aec4ada9cd077eb3f072 WatchSource:0}: Error finding container 1f459a9b667afdfb7d8e50d196ff14a570d1c40a8ba5aec4ada9cd077eb3f072: Status 404 returned error can't find the container with id 1f459a9b667afdfb7d8e50d196ff14a570d1c40a8ba5aec4ada9cd077eb3f072
Sep 11 12:09:10.948021 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:09:10.947991    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8d375e02-87b587458-7m5sf" event={"ID":"22bf031b-a2cd-4d6e-b24f-5064612ee47a","Type":"ContainerStarted","Data":"40cf01e8c5e036eca293147f60f8ec5886ff7ab301ea66ffe3b4960f4abf17e3"}
Sep 11 12:09:10.948165 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:09:10.948045    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8d375e02-87b587458-7m5sf" event={"ID":"22bf031b-a2cd-4d6e-b24f-5064612ee47a","Type":"ContainerStarted","Data":"9af53fdac25dcf6ca1785ffed826179c35218d256f39f13d9154f140aaeba099"}
Sep 11 12:09:10.948165 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:09:10.948090    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8d375e02-87b587458-7m5sf"
Sep 11 12:09:10.949954 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:09:10.949924    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-8d375e02-99f8c946-tvcq6" event={"ID":"ab9a95e3-2546-482a-a9cb-b11500fd901c","Type":"ContainerStarted","Data":"1f459a9b667afdfb7d8e50d196ff14a570d1c40a8ba5aec4ada9cd077eb3f072"}
Sep 11 12:09:10.968088 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:09:10.968028    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8d375e02-87b587458-7m5sf" podStartSLOduration=1.580011302 podStartE2EDuration="1.968010613s" totalImagesPullingTime="387.999311ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:09:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:09:10.96598386 +0000 UTC m=+1254.863425617" watchObservedRunningTime="2026-09-11 12:09:10.968010613 +0000 UTC m=+1254.865452358"
Sep 11 12:09:11.957455 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:09:11.957422    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-8d375e02-99f8c946-tvcq6" event={"ID":"ab9a95e3-2546-482a-a9cb-b11500fd901c","Type":"ContainerStarted","Data":"e6f8cdfdee28f808f8fd191d0f1d67e85351e886549b734dc80b8f6255cbf26f"}
Sep 11 12:09:11.957883 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:09:11.957501    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-8d375e02-99f8c946-tvcq6"
Sep 11 12:09:11.978142 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:09:11.978085    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-8d375e02-99f8c946-tvcq6" podStartSLOduration=1.978066439 podStartE2EDuration="1.978066439s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:09:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:09:11.975622578 +0000 UTC m=+1255.873064334" watchObservedRunningTime="2026-09-11 12:09:11.978066439 +0000 UTC m=+1255.875508183"
Sep 11 12:09:12.461683 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:09:12.461647    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:16.963272 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:09:16.963241    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8d375e02-87b587458-7m5sf"
Sep 11 12:09:22.965190 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:09:22.965157    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-8d375e02-99f8c946-tvcq6"
Sep 11 12:09:25.305746 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:09:25.305709    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:25.406043 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:09:25.405999    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:27.267089 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:09:27.267049    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:09:30.616438 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:09:30.616405    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:33.186268 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:09:33.186226    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:41.572182 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:09:41.572146    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:09:48.510636 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:09:48.510600    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:58.274099 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:09:58.274060    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:10:04.236175 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:04.236143    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:10:17.176496 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:17.176456    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:10:22.587303 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:22.587267    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8d375e02-2g48h"]
Sep 11 12:10:22.589937 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:22.589922    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8d375e02-2g48h"
Sep 11 12:10:22.593233 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:22.593201    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9xkpf\""
Sep 11 12:10:22.608130 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:22.608106    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8d375e02-2g48h"]
Sep 11 12:10:22.742799 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:22.742761    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/74fb2e57-c049-49c6-b03a-a2fbc03cfe6c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-8d375e02-2g48h\" (UID: \"74fb2e57-c049-49c6-b03a-a2fbc03cfe6c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8d375e02-2g48h"
Sep 11 12:10:22.843642 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:22.843606    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/74fb2e57-c049-49c6-b03a-a2fbc03cfe6c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-8d375e02-2g48h\" (UID: \"74fb2e57-c049-49c6-b03a-a2fbc03cfe6c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8d375e02-2g48h"
Sep 11 12:10:22.844250 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:22.844224    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/74fb2e57-c049-49c6-b03a-a2fbc03cfe6c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-8d375e02-2g48h\" (UID: \"74fb2e57-c049-49c6-b03a-a2fbc03cfe6c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8d375e02-2g48h"
Sep 11 12:10:22.900532 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:22.900493    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8d375e02-2g48h"
Sep 11 12:10:23.179724 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:23.179651    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8d375e02-2g48h"]
Sep 11 12:10:23.183571 ip-10-0-142-145 kubenswrapper[2579]: W0911 12:10:23.183539    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74fb2e57_c049_49c6_b03a_a2fbc03cfe6c.slice/crio-05715f7369a578aab519ea599b8e1f11b527d9f962a1394e759e24e6558ad83b WatchSource:0}: Error finding container 05715f7369a578aab519ea599b8e1f11b527d9f962a1394e759e24e6558ad83b: Status 404 returned error can't find the container with id 05715f7369a578aab519ea599b8e1f11b527d9f962a1394e759e24e6558ad83b
Sep 11 12:10:23.353681 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:23.353644    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8d375e02-2g48h" event={"ID":"74fb2e57-c049-49c6-b03a-a2fbc03cfe6c","Type":"ContainerStarted","Data":"05715f7369a578aab519ea599b8e1f11b527d9f962a1394e759e24e6558ad83b"}
Sep 11 12:10:24.360363 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:24.360328    2579 generic.go:356] "Generic (PLEG): container finished" podID="74fb2e57-c049-49c6-b03a-a2fbc03cfe6c" containerID="30ce195b4b161b45c69ffa3610ba8549609f6ee9b48e93beed7379455fd0b115" exitCode=0
Sep 11 12:10:24.360819 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:24.360423    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8d375e02-2g48h" event={"ID":"74fb2e57-c049-49c6-b03a-a2fbc03cfe6c","Type":"ContainerDied","Data":"30ce195b4b161b45c69ffa3610ba8549609f6ee9b48e93beed7379455fd0b115"}
Sep 11 12:10:25.503771 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:25.503746    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8d375e02-2g48h"
Sep 11 12:10:25.670724 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:25.670632    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/74fb2e57-c049-49c6-b03a-a2fbc03cfe6c-maas-api-service-ca\") pod \"74fb2e57-c049-49c6-b03a-a2fbc03cfe6c\" (UID: \"74fb2e57-c049-49c6-b03a-a2fbc03cfe6c\") "
Sep 11 12:10:25.671056 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:25.671034    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74fb2e57-c049-49c6-b03a-a2fbc03cfe6c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "74fb2e57-c049-49c6-b03a-a2fbc03cfe6c" (UID: "74fb2e57-c049-49c6-b03a-a2fbc03cfe6c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:10:25.772222 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:25.772181    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/74fb2e57-c049-49c6-b03a-a2fbc03cfe6c-maas-api-service-ca\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:10:26.373514 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:26.373472    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8d375e02-2g48h" event={"ID":"74fb2e57-c049-49c6-b03a-a2fbc03cfe6c","Type":"ContainerDied","Data":"05715f7369a578aab519ea599b8e1f11b527d9f962a1394e759e24e6558ad83b"}
Sep 11 12:10:26.373690 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:26.373530    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="05715f7369a578aab519ea599b8e1f11b527d9f962a1394e759e24e6558ad83b"
Sep 11 12:10:26.373690 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:26.373494    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8d375e02-2g48h"
Sep 11 12:10:33.167837 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:33.167801    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:10:38.246575 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:38.246540    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8d375e02-87b587458-7m5sf"]
Sep 11 12:10:38.247058 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:38.246770    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8d375e02-87b587458-7m5sf" podUID="22bf031b-a2cd-4d6e-b24f-5064612ee47a" containerName="maas-api" containerID="cri-o://40cf01e8c5e036eca293147f60f8ec5886ff7ab301ea66ffe3b4960f4abf17e3" gracePeriod=30
Sep 11 12:10:38.262634 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:38.262600    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:10:38.304866 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:38.304836    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-8d375e02-99f8c946-tvcq6"]
Sep 11 12:10:38.305063 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:38.305037    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-8d375e02-99f8c946-tvcq6" podUID="ab9a95e3-2546-482a-a9cb-b11500fd901c" containerName="payload-pre-processing" containerID="cri-o://e6f8cdfdee28f808f8fd191d0f1d67e85351e886549b734dc80b8f6255cbf26f" gracePeriod=30
Sep 11 12:10:38.445564 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:38.445229    2579 generic.go:356] "Generic (PLEG): container finished" podID="ab9a95e3-2546-482a-a9cb-b11500fd901c" containerID="e6f8cdfdee28f808f8fd191d0f1d67e85351e886549b734dc80b8f6255cbf26f" exitCode=0
Sep 11 12:10:38.445564 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:38.445495    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-8d375e02-99f8c946-tvcq6" event={"ID":"ab9a95e3-2546-482a-a9cb-b11500fd901c","Type":"ContainerDied","Data":"e6f8cdfdee28f808f8fd191d0f1d67e85351e886549b734dc80b8f6255cbf26f"}
Sep 11 12:10:38.447772 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:38.447747    2579 generic.go:356] "Generic (PLEG): container finished" podID="22bf031b-a2cd-4d6e-b24f-5064612ee47a" containerID="40cf01e8c5e036eca293147f60f8ec5886ff7ab301ea66ffe3b4960f4abf17e3" exitCode=0
Sep 11 12:10:38.447892 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:38.447784    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8d375e02-87b587458-7m5sf" event={"ID":"22bf031b-a2cd-4d6e-b24f-5064612ee47a","Type":"ContainerDied","Data":"40cf01e8c5e036eca293147f60f8ec5886ff7ab301ea66ffe3b4960f4abf17e3"}
Sep 11 12:10:38.503980 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:38.503857    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8d375e02-2g48h"]
Sep 11 12:10:38.507199 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:38.507173    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8d375e02-2g48h"]
Sep 11 12:10:38.576462 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:38.576155    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8d375e02-87b587458-7m5sf"
Sep 11 12:10:38.586635 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:38.586587    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="74fb2e57-c049-49c6-b03a-a2fbc03cfe6c" path="/var/lib/kubelet/pods/74fb2e57-c049-49c6-b03a-a2fbc03cfe6c/volumes"
Sep 11 12:10:38.593711 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:38.593683    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/22bf031b-a2cd-4d6e-b24f-5064612ee47a-openshift-service-ca\") pod \"22bf031b-a2cd-4d6e-b24f-5064612ee47a\" (UID: \"22bf031b-a2cd-4d6e-b24f-5064612ee47a\") "
Sep 11 12:10:38.593869 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:38.593729    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/22bf031b-a2cd-4d6e-b24f-5064612ee47a-maas-api-tls\") pod \"22bf031b-a2cd-4d6e-b24f-5064612ee47a\" (UID: \"22bf031b-a2cd-4d6e-b24f-5064612ee47a\") "
Sep 11 12:10:38.593869 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:38.593833    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/22bf031b-a2cd-4d6e-b24f-5064612ee47a-metrics-certs\") pod \"22bf031b-a2cd-4d6e-b24f-5064612ee47a\" (UID: \"22bf031b-a2cd-4d6e-b24f-5064612ee47a\") "
Sep 11 12:10:38.593869 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:38.593863    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7h9wl\" (UniqueName: \"kubernetes.io/projected/22bf031b-a2cd-4d6e-b24f-5064612ee47a-kube-api-access-7h9wl\") pod \"22bf031b-a2cd-4d6e-b24f-5064612ee47a\" (UID: \"22bf031b-a2cd-4d6e-b24f-5064612ee47a\") "
Sep 11 12:10:38.594128 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:38.594095    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22bf031b-a2cd-4d6e-b24f-5064612ee47a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "22bf031b-a2cd-4d6e-b24f-5064612ee47a" (UID: "22bf031b-a2cd-4d6e-b24f-5064612ee47a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:10:38.594220 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:38.594186    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/22bf031b-a2cd-4d6e-b24f-5064612ee47a-openshift-service-ca\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:10:38.596209 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:38.596175    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22bf031b-a2cd-4d6e-b24f-5064612ee47a-kube-api-access-7h9wl" (OuterVolumeSpecName: "kube-api-access-7h9wl") pod "22bf031b-a2cd-4d6e-b24f-5064612ee47a" (UID: "22bf031b-a2cd-4d6e-b24f-5064612ee47a"). InnerVolumeSpecName "kube-api-access-7h9wl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:10:38.596209 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:38.596183    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22bf031b-a2cd-4d6e-b24f-5064612ee47a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "22bf031b-a2cd-4d6e-b24f-5064612ee47a" (UID: "22bf031b-a2cd-4d6e-b24f-5064612ee47a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:10:38.596341 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:38.596238    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22bf031b-a2cd-4d6e-b24f-5064612ee47a-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "22bf031b-a2cd-4d6e-b24f-5064612ee47a" (UID: "22bf031b-a2cd-4d6e-b24f-5064612ee47a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:10:38.601941 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:38.601914    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-8d375e02-99f8c946-tvcq6"
Sep 11 12:10:38.694737 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:38.694695    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mbp4f\" (UniqueName: \"kubernetes.io/projected/ab9a95e3-2546-482a-a9cb-b11500fd901c-kube-api-access-mbp4f\") pod \"ab9a95e3-2546-482a-a9cb-b11500fd901c\" (UID: \"ab9a95e3-2546-482a-a9cb-b11500fd901c\") "
Sep 11 12:10:38.694906 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:38.694818    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab9a95e3-2546-482a-a9cb-b11500fd901c-plugins-config-volume\") pod \"ab9a95e3-2546-482a-a9cb-b11500fd901c\" (UID: \"ab9a95e3-2546-482a-a9cb-b11500fd901c\") "
Sep 11 12:10:38.695144 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:38.695121    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7h9wl\" (UniqueName: \"kubernetes.io/projected/22bf031b-a2cd-4d6e-b24f-5064612ee47a-kube-api-access-7h9wl\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:10:38.695144 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:38.695145    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/22bf031b-a2cd-4d6e-b24f-5064612ee47a-maas-api-tls\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:10:38.695294 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:38.695155    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/22bf031b-a2cd-4d6e-b24f-5064612ee47a-metrics-certs\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:10:38.695294 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:38.695202    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab9a95e3-2546-482a-a9cb-b11500fd901c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ab9a95e3-2546-482a-a9cb-b11500fd901c" (UID: "ab9a95e3-2546-482a-a9cb-b11500fd901c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:10:38.696768 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:38.696749    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab9a95e3-2546-482a-a9cb-b11500fd901c-kube-api-access-mbp4f" (OuterVolumeSpecName: "kube-api-access-mbp4f") pod "ab9a95e3-2546-482a-a9cb-b11500fd901c" (UID: "ab9a95e3-2546-482a-a9cb-b11500fd901c"). InnerVolumeSpecName "kube-api-access-mbp4f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:10:38.796477 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:38.796322    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mbp4f\" (UniqueName: \"kubernetes.io/projected/ab9a95e3-2546-482a-a9cb-b11500fd901c-kube-api-access-mbp4f\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:10:38.796477 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:38.796354    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab9a95e3-2546-482a-a9cb-b11500fd901c-plugins-config-volume\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:10:39.454376 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:39.454347    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8d375e02-87b587458-7m5sf"
Sep 11 12:10:39.454376 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:39.454365    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8d375e02-87b587458-7m5sf" event={"ID":"22bf031b-a2cd-4d6e-b24f-5064612ee47a","Type":"ContainerDied","Data":"9af53fdac25dcf6ca1785ffed826179c35218d256f39f13d9154f140aaeba099"}
Sep 11 12:10:39.454885 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:39.454449    2579 scope.go:117] "RemoveContainer" containerID="40cf01e8c5e036eca293147f60f8ec5886ff7ab301ea66ffe3b4960f4abf17e3"
Sep 11 12:10:39.455976 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:39.455897    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-8d375e02-99f8c946-tvcq6"
Sep 11 12:10:39.455976 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:39.455939    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-8d375e02-99f8c946-tvcq6" event={"ID":"ab9a95e3-2546-482a-a9cb-b11500fd901c","Type":"ContainerDied","Data":"1f459a9b667afdfb7d8e50d196ff14a570d1c40a8ba5aec4ada9cd077eb3f072"}
Sep 11 12:10:39.464946 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:39.464927    2579 scope.go:117] "RemoveContainer" containerID="e6f8cdfdee28f808f8fd191d0f1d67e85351e886549b734dc80b8f6255cbf26f"
Sep 11 12:10:39.480983 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:39.480949    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8d375e02-87b587458-7m5sf"]
Sep 11 12:10:39.485357 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:39.485330    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8d375e02-87b587458-7m5sf"]
Sep 11 12:10:39.496951 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:39.496924    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-8d375e02-99f8c946-tvcq6"]
Sep 11 12:10:39.501323 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:39.501300    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-8d375e02-99f8c946-tvcq6"]
Sep 11 12:10:40.576827 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:40.576794    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="22bf031b-a2cd-4d6e-b24f-5064612ee47a" path="/var/lib/kubelet/pods/22bf031b-a2cd-4d6e-b24f-5064612ee47a/volumes"
Sep 11 12:10:40.577202 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:40.577159    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab9a95e3-2546-482a-a9cb-b11500fd901c" path="/var/lib/kubelet/pods/ab9a95e3-2546-482a-a9cb-b11500fd901c/volumes"
Sep 11 12:10:40.948085 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:40.948045    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:10:41.900783 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:41.900753    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:10:46.634095 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:46.634064    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:10:51.395826 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:10:51.395791    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:11:06.499244 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:11:06.499205    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:11:21.045912 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:11:21.045872    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:11:21.456873 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:11:21.456833    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:11:22.052599 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:11:22.052563    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:11:22.833541 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:11:22.833503    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:11:23.968451 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:11:23.968384    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:11:27.675217 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:11:27.675183    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:11:30.132172 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:11:30.132134    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:11:30.231315 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:11:30.231276    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:11:30.513919 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:11:30.513813    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:11:30.623040 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:11:30.623002    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:11:31.660988 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:11:31.660932    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:11:31.807471 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:11:31.807439    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:11:32.031667 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:11:32.031569    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:11:32.553216 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:11:32.553177    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:11:32.640985 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:11:32.640950    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:11:33.989205 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:11:33.989166    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:11:40.370929 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:11:40.370894    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:11:40.470843 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:11:40.470807    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:11:41.058454 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:11:41.058423    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:11:45.595027 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:11:45.594995    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:11:47.845420 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:11:47.845362    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:11:47.915821 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:11:47.915782    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:11:48.040558 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:11:48.040512    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:11:50.956989 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:11:50.956926    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:11:51.057991 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:11:51.057960    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:11:51.764829 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:11:51.764796    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:11:59.406628 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:11:59.406591    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:11:59.539991 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:11:59.539958    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:12:06.465918 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:06.465876    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:12:09.780487 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:09.780385    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:12:16.690083 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:16.690053    2579 scope.go:117] "RemoveContainer" containerID="4fb76d47e3354f7a699294551c0c13eb42109f65312d06d28dae8f6f611efd69"
Sep 11 12:12:19.690211 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:19.690176    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:12:25.940008 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:25.939964    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:12:26.032786 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:26.032750    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:12:26.264892 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:26.264793    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:12:26.545329 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:26.545242    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg"]
Sep 11 12:12:26.545744 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:26.545728    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74fb2e57-c049-49c6-b03a-a2fbc03cfe6c" containerName="revoke-keys"
Sep 11 12:12:26.545814 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:26.545746    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="74fb2e57-c049-49c6-b03a-a2fbc03cfe6c" containerName="revoke-keys"
Sep 11 12:12:26.545814 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:26.545766    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab9a95e3-2546-482a-a9cb-b11500fd901c" containerName="payload-pre-processing"
Sep 11 12:12:26.545814 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:26.545774    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab9a95e3-2546-482a-a9cb-b11500fd901c" containerName="payload-pre-processing"
Sep 11 12:12:26.545814 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:26.545788    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22bf031b-a2cd-4d6e-b24f-5064612ee47a" containerName="maas-api"
Sep 11 12:12:26.545814 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:26.545794    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="22bf031b-a2cd-4d6e-b24f-5064612ee47a" containerName="maas-api"
Sep 11 12:12:26.546102 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:26.545869    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="22bf031b-a2cd-4d6e-b24f-5064612ee47a" containerName="maas-api"
Sep 11 12:12:26.546102 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:26.545881    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="74fb2e57-c049-49c6-b03a-a2fbc03cfe6c" containerName="revoke-keys"
Sep 11 12:12:26.546102 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:26.545888    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab9a95e3-2546-482a-a9cb-b11500fd901c" containerName="payload-pre-processing"
Sep 11 12:12:26.550424 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:26.550405    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg"
Sep 11 12:12:26.553153 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:26.553128    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-fb3c56-openshift-default-dockercfg-zn8w5\""
Sep 11 12:12:26.562864 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:26.562828    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg"]
Sep 11 12:12:26.587517 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:26.587456    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:12:26.655924 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:26.655881    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a319527-8f12-4d0d-8609-3d15dc436d72-workload-socket\") pod \"e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg\" (UID: \"0a319527-8f12-4d0d-8609-3d15dc436d72\") " pod="openshift-ingress/e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg"
Sep 11 12:12:26.655924 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:26.655928    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a319527-8f12-4d0d-8609-3d15dc436d72-istio-podinfo\") pod \"e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg\" (UID: \"0a319527-8f12-4d0d-8609-3d15dc436d72\") " pod="openshift-ingress/e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg"
Sep 11 12:12:26.656187 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:26.655967    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a319527-8f12-4d0d-8609-3d15dc436d72-istiod-ca-cert\") pod \"e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg\" (UID: \"0a319527-8f12-4d0d-8609-3d15dc436d72\") " pod="openshift-ingress/e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg"
Sep 11 12:12:26.656187 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:26.656038    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a319527-8f12-4d0d-8609-3d15dc436d72-istio-token\") pod \"e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg\" (UID: \"0a319527-8f12-4d0d-8609-3d15dc436d72\") " pod="openshift-ingress/e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg"
Sep 11 12:12:26.656187 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:26.656069    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftcm4\" (UniqueName: \"kubernetes.io/projected/0a319527-8f12-4d0d-8609-3d15dc436d72-kube-api-access-ftcm4\") pod \"e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg\" (UID: \"0a319527-8f12-4d0d-8609-3d15dc436d72\") " pod="openshift-ingress/e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg"
Sep 11 12:12:26.656187 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:26.656103    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a319527-8f12-4d0d-8609-3d15dc436d72-istio-data\") pod \"e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg\" (UID: \"0a319527-8f12-4d0d-8609-3d15dc436d72\") " pod="openshift-ingress/e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg"
Sep 11 12:12:26.656187 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:26.656135    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a319527-8f12-4d0d-8609-3d15dc436d72-istio-envoy\") pod \"e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg\" (UID: \"0a319527-8f12-4d0d-8609-3d15dc436d72\") " pod="openshift-ingress/e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg"
Sep 11 12:12:26.656187 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:26.656171    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a319527-8f12-4d0d-8609-3d15dc436d72-credential-socket\") pod \"e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg\" (UID: \"0a319527-8f12-4d0d-8609-3d15dc436d72\") " pod="openshift-ingress/e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg"
Sep 11 12:12:26.656423 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:26.656251    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a319527-8f12-4d0d-8609-3d15dc436d72-workload-certs\") pod \"e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg\" (UID: \"0a319527-8f12-4d0d-8609-3d15dc436d72\") " pod="openshift-ingress/e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg"
Sep 11 12:12:26.757645 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:26.757613    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a319527-8f12-4d0d-8609-3d15dc436d72-workload-socket\") pod \"e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg\" (UID: \"0a319527-8f12-4d0d-8609-3d15dc436d72\") " pod="openshift-ingress/e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg"
Sep 11 12:12:26.757850 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:26.757658    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a319527-8f12-4d0d-8609-3d15dc436d72-istio-podinfo\") pod \"e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg\" (UID: \"0a319527-8f12-4d0d-8609-3d15dc436d72\") " pod="openshift-ingress/e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg"
Sep 11 12:12:26.757850 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:26.757678    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a319527-8f12-4d0d-8609-3d15dc436d72-istiod-ca-cert\") pod \"e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg\" (UID: \"0a319527-8f12-4d0d-8609-3d15dc436d72\") " pod="openshift-ingress/e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg"
Sep 11 12:12:26.757850 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:26.757715    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a319527-8f12-4d0d-8609-3d15dc436d72-istio-token\") pod \"e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg\" (UID: \"0a319527-8f12-4d0d-8609-3d15dc436d72\") " pod="openshift-ingress/e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg"
Sep 11 12:12:26.757850 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:26.757738    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftcm4\" (UniqueName: \"kubernetes.io/projected/0a319527-8f12-4d0d-8609-3d15dc436d72-kube-api-access-ftcm4\") pod \"e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg\" (UID: \"0a319527-8f12-4d0d-8609-3d15dc436d72\") " pod="openshift-ingress/e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg"
Sep 11 12:12:26.757850 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:26.757759    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a319527-8f12-4d0d-8609-3d15dc436d72-istio-data\") pod \"e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg\" (UID: \"0a319527-8f12-4d0d-8609-3d15dc436d72\") " pod="openshift-ingress/e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg"
Sep 11 12:12:26.757850 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:26.757788    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a319527-8f12-4d0d-8609-3d15dc436d72-istio-envoy\") pod \"e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg\" (UID: \"0a319527-8f12-4d0d-8609-3d15dc436d72\") " pod="openshift-ingress/e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg"
Sep 11 12:12:26.757850 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:26.757821    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a319527-8f12-4d0d-8609-3d15dc436d72-credential-socket\") pod \"e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg\" (UID: \"0a319527-8f12-4d0d-8609-3d15dc436d72\") " pod="openshift-ingress/e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg"
Sep 11 12:12:26.757850 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:26.757850    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a319527-8f12-4d0d-8609-3d15dc436d72-workload-certs\") pod \"e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg\" (UID: \"0a319527-8f12-4d0d-8609-3d15dc436d72\") " pod="openshift-ingress/e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg"
Sep 11 12:12:26.758306 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:26.758122    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a319527-8f12-4d0d-8609-3d15dc436d72-workload-socket\") pod \"e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg\" (UID: \"0a319527-8f12-4d0d-8609-3d15dc436d72\") " pod="openshift-ingress/e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg"
Sep 11 12:12:26.758306 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:26.758147    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a319527-8f12-4d0d-8609-3d15dc436d72-istio-data\") pod \"e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg\" (UID: \"0a319527-8f12-4d0d-8609-3d15dc436d72\") " pod="openshift-ingress/e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg"
Sep 11 12:12:26.758306 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:26.758245    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a319527-8f12-4d0d-8609-3d15dc436d72-workload-certs\") pod \"e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg\" (UID: \"0a319527-8f12-4d0d-8609-3d15dc436d72\") " pod="openshift-ingress/e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg"
Sep 11 12:12:26.758479 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:26.758381    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a319527-8f12-4d0d-8609-3d15dc436d72-credential-socket\") pod \"e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg\" (UID: \"0a319527-8f12-4d0d-8609-3d15dc436d72\") " pod="openshift-ingress/e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg"
Sep 11 12:12:26.758528 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:26.758516    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a319527-8f12-4d0d-8609-3d15dc436d72-istiod-ca-cert\") pod \"e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg\" (UID: \"0a319527-8f12-4d0d-8609-3d15dc436d72\") " pod="openshift-ingress/e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg"
Sep 11 12:12:26.760732 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:26.760709    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a319527-8f12-4d0d-8609-3d15dc436d72-istio-envoy\") pod \"e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg\" (UID: \"0a319527-8f12-4d0d-8609-3d15dc436d72\") " pod="openshift-ingress/e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg"
Sep 11 12:12:26.760897 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:26.760881    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a319527-8f12-4d0d-8609-3d15dc436d72-istio-podinfo\") pod \"e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg\" (UID: \"0a319527-8f12-4d0d-8609-3d15dc436d72\") " pod="openshift-ingress/e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg"
Sep 11 12:12:26.778128 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:26.778106    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a319527-8f12-4d0d-8609-3d15dc436d72-istio-token\") pod \"e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg\" (UID: \"0a319527-8f12-4d0d-8609-3d15dc436d72\") " pod="openshift-ingress/e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg"
Sep 11 12:12:26.778325 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:26.778303    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftcm4\" (UniqueName: \"kubernetes.io/projected/0a319527-8f12-4d0d-8609-3d15dc436d72-kube-api-access-ftcm4\") pod \"e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg\" (UID: \"0a319527-8f12-4d0d-8609-3d15dc436d72\") " pod="openshift-ingress/e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg"
Sep 11 12:12:26.864004 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:26.863965    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg"
Sep 11 12:12:26.913885 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:26.913849    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:12:27.153201 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:27.153174    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg"]
Sep 11 12:12:27.155912 ip-10-0-142-145 kubenswrapper[2579]: W0911 12:12:27.155880    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a319527_8f12_4d0d_8609_3d15dc436d72.slice/crio-a6f7381fdec585b81a69bf7befcc2e57343da315205ce6f6350519636875f6e7 WatchSource:0}: Error finding container a6f7381fdec585b81a69bf7befcc2e57343da315205ce6f6350519636875f6e7: Status 404 returned error can't find the container with id a6f7381fdec585b81a69bf7befcc2e57343da315205ce6f6350519636875f6e7
Sep 11 12:12:27.158224 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:27.158193    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 12:12:27.158298 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:27.158257    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 12:12:27.158298 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:27.158286    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 12:12:28.052097 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:28.052057    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg" event={"ID":"0a319527-8f12-4d0d-8609-3d15dc436d72","Type":"ContainerStarted","Data":"ca8d71d803faa1578dc8a0d50b6fc2d3a4aecf182c772509db0176885ea32957"}
Sep 11 12:12:28.052302 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:28.052112    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg" event={"ID":"0a319527-8f12-4d0d-8609-3d15dc436d72","Type":"ContainerStarted","Data":"a6f7381fdec585b81a69bf7befcc2e57343da315205ce6f6350519636875f6e7"}
Sep 11 12:12:28.071140 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:28.071111    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:12:28.076168 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:28.076109    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg" podStartSLOduration=2.0760898230000002 podStartE2EDuration="2.076089823s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:12:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:12:28.0739974 +0000 UTC m=+1451.971439156" watchObservedRunningTime="2026-09-11 12:12:28.076089823 +0000 UTC m=+1451.973531570"
Sep 11 12:12:28.865054 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:28.865012    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg"
Sep 11 12:12:28.871336 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:28.871307    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg"
Sep 11 12:12:29.058140 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:29.058108    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg"
Sep 11 12:12:29.059288 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:29.059266    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg"
Sep 11 12:12:29.244090 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:29.244007    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:12:30.054135 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:30.054095    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:12:30.159146 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:30.159118    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:12:30.559720 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:30.559680    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:12:31.422674 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:31.422635    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:12:31.524652 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:31.524609    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:12:31.621938 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:31.621898    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:12:32.234698 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:32.234657    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-fb3c56-d8c7bd578-l7nz4"]
Sep 11 12:12:32.238575 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:32.238555    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-fb3c56-d8c7bd578-l7nz4"
Sep 11 12:12:32.243881 ip-10-0-142-145 kubenswrapper[2579]: E0911 12:12:32.243849    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"payload-pre-processing-e2e-api-a-fb3c56-d8c7bd578-l7nz4\" is forbidden: User \"system:node:ip-10-0-142-145.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-142-145.ec2.internal' and this object" podUID="48e1009c-d1cb-46a1-8a70-22efe471beb0" pod="openshift-ingress/payload-pre-processing-e2e-api-a-fb3c56-d8c7bd578-l7nz4"
Sep 11 12:12:32.243881 ip-10-0-142-145 kubenswrapper[2579]: E0911 12:12:32.243871    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"payload-processing-e2e-api-a-fb3c56-dockercfg-6qfzk\" is forbidden: User \"system:node:ip-10-0-142-145.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-142-145.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-fb3c56-dockercfg-6qfzk\"" type="*v1.Secret"
Sep 11 12:12:32.244442 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:32.244420    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-fb3c56\""
Sep 11 12:12:32.254685 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:32.254647    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-fb3c56-d8c7bd578-l7nz4"]
Sep 11 12:12:32.412574 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:32.412542    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/48e1009c-d1cb-46a1-8a70-22efe471beb0-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-fb3c56-d8c7bd578-l7nz4\" (UID: \"48e1009c-d1cb-46a1-8a70-22efe471beb0\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-fb3c56-d8c7bd578-l7nz4"
Sep 11 12:12:32.412757 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:32.412608    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xh7rz\" (UniqueName: \"kubernetes.io/projected/48e1009c-d1cb-46a1-8a70-22efe471beb0-kube-api-access-xh7rz\") pod \"payload-pre-processing-e2e-api-a-fb3c56-d8c7bd578-l7nz4\" (UID: \"48e1009c-d1cb-46a1-8a70-22efe471beb0\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-fb3c56-d8c7bd578-l7nz4"
Sep 11 12:12:32.513985 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:32.513888    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xh7rz\" (UniqueName: \"kubernetes.io/projected/48e1009c-d1cb-46a1-8a70-22efe471beb0-kube-api-access-xh7rz\") pod \"payload-pre-processing-e2e-api-a-fb3c56-d8c7bd578-l7nz4\" (UID: \"48e1009c-d1cb-46a1-8a70-22efe471beb0\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-fb3c56-d8c7bd578-l7nz4"
Sep 11 12:12:32.514364 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:32.514000    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/48e1009c-d1cb-46a1-8a70-22efe471beb0-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-fb3c56-d8c7bd578-l7nz4\" (UID: \"48e1009c-d1cb-46a1-8a70-22efe471beb0\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-fb3c56-d8c7bd578-l7nz4"
Sep 11 12:12:32.514770 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:32.514749    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/48e1009c-d1cb-46a1-8a70-22efe471beb0-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-fb3c56-d8c7bd578-l7nz4\" (UID: \"48e1009c-d1cb-46a1-8a70-22efe471beb0\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-fb3c56-d8c7bd578-l7nz4"
Sep 11 12:12:32.524708 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:32.524686    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xh7rz\" (UniqueName: \"kubernetes.io/projected/48e1009c-d1cb-46a1-8a70-22efe471beb0-kube-api-access-xh7rz\") pod \"payload-pre-processing-e2e-api-a-fb3c56-d8c7bd578-l7nz4\" (UID: \"48e1009c-d1cb-46a1-8a70-22efe471beb0\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-fb3c56-d8c7bd578-l7nz4"
Sep 11 12:12:33.330501 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:33.330468    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:12:33.433136 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:33.433096    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:12:33.553170 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:33.553123    2579 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-e2e-api-a-fb3c56-d8c7bd578-l7nz4" secret="" err="failed to sync secret cache: timed out waiting for the condition"
Sep 11 12:12:33.553691 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:33.553209    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-fb3c56-d8c7bd578-l7nz4"
Sep 11 12:12:33.586077 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:33.585989    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-fb3c56-dockercfg-6qfzk\""
Sep 11 12:12:33.827779 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:33.827756    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-fb3c56-d8c7bd578-l7nz4"]
Sep 11 12:12:33.829927 ip-10-0-142-145 kubenswrapper[2579]: W0911 12:12:33.829893    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48e1009c_d1cb_46a1_8a70_22efe471beb0.slice/crio-8ab47f28031981c899935e235d6e9aaf7ab348f9e0c76f73cd5cf1b96ed38e5b WatchSource:0}: Error finding container 8ab47f28031981c899935e235d6e9aaf7ab348f9e0c76f73cd5cf1b96ed38e5b: Status 404 returned error can't find the container with id 8ab47f28031981c899935e235d6e9aaf7ab348f9e0c76f73cd5cf1b96ed38e5b
Sep 11 12:12:34.089300 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:34.089217    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-fb3c56-d8c7bd578-l7nz4" event={"ID":"48e1009c-d1cb-46a1-8a70-22efe471beb0","Type":"ContainerStarted","Data":"a4342186b2dfb69811cf24ae02f43fd6a9bf32dce030337bcd88910247b6c017"}
Sep 11 12:12:34.089300 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:34.089279    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-fb3c56-d8c7bd578-l7nz4" event={"ID":"48e1009c-d1cb-46a1-8a70-22efe471beb0","Type":"ContainerStarted","Data":"8ab47f28031981c899935e235d6e9aaf7ab348f9e0c76f73cd5cf1b96ed38e5b"}
Sep 11 12:12:34.089524 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:34.089326    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-fb3c56-d8c7bd578-l7nz4"
Sep 11 12:12:34.108286 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:34.108234    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-fb3c56-d8c7bd578-l7nz4" podStartSLOduration=2.108217872 podStartE2EDuration="2.108217872s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:12:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:12:34.107574595 +0000 UTC m=+1458.005016338" watchObservedRunningTime="2026-09-11 12:12:34.108217872 +0000 UTC m=+1458.005659617"
Sep 11 12:12:43.583344 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:43.583306    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:12:43.957687 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:43.957655    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:12:45.096824 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:45.096788    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-fb3c56-d8c7bd578-l7nz4"
Sep 11 12:12:45.099540 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:45.099506    2579 patch_prober.go:28] interesting pod/e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 11 12:12:45.099642 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:45.099616    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg" podUID="0a319527-8f12-4d0d-8609-3d15dc436d72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 11 12:12:46.156308 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:46.156274    2579 generic.go:356] "Generic (PLEG): container finished" podID="0a319527-8f12-4d0d-8609-3d15dc436d72" containerID="ca8d71d803faa1578dc8a0d50b6fc2d3a4aecf182c772509db0176885ea32957" exitCode=0
Sep 11 12:12:46.156867 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:46.156348    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg" event={"ID":"0a319527-8f12-4d0d-8609-3d15dc436d72","Type":"ContainerDied","Data":"ca8d71d803faa1578dc8a0d50b6fc2d3a4aecf182c772509db0176885ea32957"}
Sep 11 12:12:46.156941 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:46.156872    2579 scope.go:117] "RemoveContainer" containerID="ca8d71d803faa1578dc8a0d50b6fc2d3a4aecf182c772509db0176885ea32957"
Sep 11 12:12:46.157975 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:46.157941    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 12:12:46.158071 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:46.158015    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 12:12:46.158071 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:46.158056    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 12:12:47.164077 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:47.164044    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg" event={"ID":"0a319527-8f12-4d0d-8609-3d15dc436d72","Type":"ContainerStarted","Data":"fd9819bc1b0f407c0a14e3297ff2b86242b06beb343189159b007384568a04cb"}
Sep 11 12:12:47.864478 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:47.864426    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg"
Sep 11 12:12:47.864795 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:47.864724    2579 patch_prober.go:28] interesting pod/e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Sep 11 12:12:47.864884 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:47.864796    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg" podUID="0a319527-8f12-4d0d-8609-3d15dc436d72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Sep 11 12:12:48.864666 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:48.864630    2579 patch_prober.go:28] interesting pod/e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Sep 11 12:12:48.865054 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:48.864685    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg" podUID="0a319527-8f12-4d0d-8609-3d15dc436d72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Sep 11 12:12:49.865324 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:49.865290    2579 patch_prober.go:28] interesting pod/e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Sep 11 12:12:49.865715 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:49.865347    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg" podUID="0a319527-8f12-4d0d-8609-3d15dc436d72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Sep 11 12:12:50.009561 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:50.009520    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:12:50.864912 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:50.864873    2579 patch_prober.go:28] interesting pod/e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Sep 11 12:12:50.865106 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:50.864941    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg" podUID="0a319527-8f12-4d0d-8609-3d15dc436d72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Sep 11 12:12:51.865167 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:51.865126    2579 patch_prober.go:28] interesting pod/e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Sep 11 12:12:51.865682 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:51.865195    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg" podUID="0a319527-8f12-4d0d-8609-3d15dc436d72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Sep 11 12:12:52.524599 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:52.524561    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:12:52.868025 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:52.867990    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg"
Sep 11 12:12:52.868493 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:52.868481    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg"
Sep 11 12:12:52.869265 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:52.869247    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg"
Sep 11 12:12:54.264986 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:54.264952    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:12:54.980659 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:54.980626    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:12:55.459529 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:55.459495    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:12:55.854797 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:55.854766    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:12:56.039796 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:56.039758    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:12:59.380279 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:59.380237    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:12:59.481282 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:12:59.481247    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:13:00.636418 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:00.636358    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:13:00.746445 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:00.746404    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:13:00.836960 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:00.836927    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:13:01.145447 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:01.145375    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:13:01.239740 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:01.239708    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:13:01.269714 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:01.269671    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-f4d189-86bd5948b9-92cgn"]
Sep 11 12:13:01.274972 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:01.274949    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-f4d189-86bd5948b9-92cgn"
Sep 11 12:13:01.277811 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:01.277786    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-f4d189-dockercfg-cv2cq\""
Sep 11 12:13:01.277982 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:01.277791    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-f4d189\""
Sep 11 12:13:01.288130 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:01.288102    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-f4d189-86bd5948b9-92cgn"]
Sep 11 12:13:01.422121 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:01.422029    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6de1d37e-5930-45f4-8c73-b923133129a5-plugins-config-volume\") pod \"payload-processing-e2e-api-b-f4d189-86bd5948b9-92cgn\" (UID: \"6de1d37e-5930-45f4-8c73-b923133129a5\") " pod="openshift-ingress/payload-processing-e2e-api-b-f4d189-86bd5948b9-92cgn"
Sep 11 12:13:01.422284 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:01.422164    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8l4r\" (UniqueName: \"kubernetes.io/projected/6de1d37e-5930-45f4-8c73-b923133129a5-kube-api-access-w8l4r\") pod \"payload-processing-e2e-api-b-f4d189-86bd5948b9-92cgn\" (UID: \"6de1d37e-5930-45f4-8c73-b923133129a5\") " pod="openshift-ingress/payload-processing-e2e-api-b-f4d189-86bd5948b9-92cgn"
Sep 11 12:13:01.522923 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:01.522882    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6de1d37e-5930-45f4-8c73-b923133129a5-plugins-config-volume\") pod \"payload-processing-e2e-api-b-f4d189-86bd5948b9-92cgn\" (UID: \"6de1d37e-5930-45f4-8c73-b923133129a5\") " pod="openshift-ingress/payload-processing-e2e-api-b-f4d189-86bd5948b9-92cgn"
Sep 11 12:13:01.523095 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:01.522975    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8l4r\" (UniqueName: \"kubernetes.io/projected/6de1d37e-5930-45f4-8c73-b923133129a5-kube-api-access-w8l4r\") pod \"payload-processing-e2e-api-b-f4d189-86bd5948b9-92cgn\" (UID: \"6de1d37e-5930-45f4-8c73-b923133129a5\") " pod="openshift-ingress/payload-processing-e2e-api-b-f4d189-86bd5948b9-92cgn"
Sep 11 12:13:01.523736 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:01.523715    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6de1d37e-5930-45f4-8c73-b923133129a5-plugins-config-volume\") pod \"payload-processing-e2e-api-b-f4d189-86bd5948b9-92cgn\" (UID: \"6de1d37e-5930-45f4-8c73-b923133129a5\") " pod="openshift-ingress/payload-processing-e2e-api-b-f4d189-86bd5948b9-92cgn"
Sep 11 12:13:01.538876 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:01.538847    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8l4r\" (UniqueName: \"kubernetes.io/projected/6de1d37e-5930-45f4-8c73-b923133129a5-kube-api-access-w8l4r\") pod \"payload-processing-e2e-api-b-f4d189-86bd5948b9-92cgn\" (UID: \"6de1d37e-5930-45f4-8c73-b923133129a5\") " pod="openshift-ingress/payload-processing-e2e-api-b-f4d189-86bd5948b9-92cgn"
Sep 11 12:13:01.588931 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:01.588874    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-f4d189-86bd5948b9-92cgn"
Sep 11 12:13:01.870317 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:01.870281    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-f4d189-86bd5948b9-92cgn"]
Sep 11 12:13:01.872690 ip-10-0-142-145 kubenswrapper[2579]: W0911 12:13:01.872659    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6de1d37e_5930_45f4_8c73_b923133129a5.slice/crio-5bf7ddd84bc3345d04def17eb4067649874228097e9bc8ea347a0b7f65a85cd9 WatchSource:0}: Error finding container 5bf7ddd84bc3345d04def17eb4067649874228097e9bc8ea347a0b7f65a85cd9: Status 404 returned error can't find the container with id 5bf7ddd84bc3345d04def17eb4067649874228097e9bc8ea347a0b7f65a85cd9
Sep 11 12:13:02.258209 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:02.258120    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-f4d189-86bd5948b9-92cgn" event={"ID":"6de1d37e-5930-45f4-8c73-b923133129a5","Type":"ContainerStarted","Data":"1f3a9a9718ed4badbe9a55870d5195a7e4dff6475521ebd6f9208f0db6ac4eab"}
Sep 11 12:13:02.258209 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:02.258184    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-f4d189-86bd5948b9-92cgn" event={"ID":"6de1d37e-5930-45f4-8c73-b923133129a5","Type":"ContainerStarted","Data":"5bf7ddd84bc3345d04def17eb4067649874228097e9bc8ea347a0b7f65a85cd9"}
Sep 11 12:13:02.258420 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:02.258241    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-f4d189-86bd5948b9-92cgn"
Sep 11 12:13:02.280825 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:02.280765    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-f4d189-86bd5948b9-92cgn" podStartSLOduration=1.280745698 podStartE2EDuration="1.280745698s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:13:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:13:02.279713744 +0000 UTC m=+1486.177155490" watchObservedRunningTime="2026-09-11 12:13:02.280745698 +0000 UTC m=+1486.178187443"
Sep 11 12:13:12.540813 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:12.540770    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:13:13.266381 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:13.266350    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-f4d189-86bd5948b9-92cgn"
Sep 11 12:13:16.614292 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:16.614261    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-ch96n_1f0f9fa5-cbd1-468b-ae81-82b48c101d37/console-operator/2.log"
Sep 11 12:13:16.614721 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:16.614643    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-ch96n_1f0f9fa5-cbd1-468b-ae81-82b48c101d37/console-operator/2.log"
Sep 11 12:13:16.857231 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:16.857188    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:13:22.657683 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:22.657650    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:13:22.776042 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:22.776005    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:13:23.612419 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:23.612362    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:13:24.303614 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:24.303584    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:13:24.388659 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:24.388627    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:13:27.047784 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:27.047749    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:13:27.084202 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:27.084168    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-fb3c56-m8zzg"]
Sep 11 12:13:27.088401 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:27.088373    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-fb3c56-m8zzg"
Sep 11 12:13:27.091383 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:27.091360    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 12:13:27.091509 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:27.091475    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9xkpf\""
Sep 11 12:13:27.095779 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:27.095754    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-fb3c56-m8zzg"]
Sep 11 12:13:27.188280 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:27.188245    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/de56150e-d7da-47da-8a86-1db61a99fd1d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-fb3c56-m8zzg\" (UID: \"de56150e-d7da-47da-8a86-1db61a99fd1d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-fb3c56-m8zzg"
Sep 11 12:13:27.240097 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:27.240064    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:13:27.289144 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:27.289108    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/de56150e-d7da-47da-8a86-1db61a99fd1d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-fb3c56-m8zzg\" (UID: \"de56150e-d7da-47da-8a86-1db61a99fd1d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-fb3c56-m8zzg"
Sep 11 12:13:27.289780 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:27.289761    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/de56150e-d7da-47da-8a86-1db61a99fd1d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-fb3c56-m8zzg\" (UID: \"de56150e-d7da-47da-8a86-1db61a99fd1d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-fb3c56-m8zzg"
Sep 11 12:13:27.400981 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:27.400941    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-fb3c56-m8zzg"
Sep 11 12:13:27.676918 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:27.676885    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-fb3c56-m8zzg"]
Sep 11 12:13:27.679204 ip-10-0-142-145 kubenswrapper[2579]: W0911 12:13:27.679172    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde56150e_d7da_47da_8a86_1db61a99fd1d.slice/crio-67d74a5ae339e9aedbb5052ce8b89c5cb06a8897cbbf07a2a0bda279e471f6bb WatchSource:0}: Error finding container 67d74a5ae339e9aedbb5052ce8b89c5cb06a8897cbbf07a2a0bda279e471f6bb: Status 404 returned error can't find the container with id 67d74a5ae339e9aedbb5052ce8b89c5cb06a8897cbbf07a2a0bda279e471f6bb
Sep 11 12:13:28.408273 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:28.408236    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-fb3c56-m8zzg" event={"ID":"de56150e-d7da-47da-8a86-1db61a99fd1d","Type":"ContainerStarted","Data":"bc17a6dd94034892bd2229cdf97a43bcef06e86a98418d1bccccc561e4bb4e5a"}
Sep 11 12:13:28.408690 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:28.408317    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-fb3c56-m8zzg" event={"ID":"de56150e-d7da-47da-8a86-1db61a99fd1d","Type":"ContainerStarted","Data":"67d74a5ae339e9aedbb5052ce8b89c5cb06a8897cbbf07a2a0bda279e471f6bb"}
Sep 11 12:13:28.429097 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:28.429042    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-fb3c56-m8zzg" podStartSLOduration=1.4290278779999999 podStartE2EDuration="1.429027878s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:13:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:13:28.427864255 +0000 UTC m=+1512.325305997" watchObservedRunningTime="2026-09-11 12:13:28.429027878 +0000 UTC m=+1512.326469623"
Sep 11 12:13:29.415615 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:29.415579    2579 generic.go:356] "Generic (PLEG): container finished" podID="de56150e-d7da-47da-8a86-1db61a99fd1d" containerID="bc17a6dd94034892bd2229cdf97a43bcef06e86a98418d1bccccc561e4bb4e5a" exitCode=0
Sep 11 12:13:29.415988 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:29.415666    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-fb3c56-m8zzg" event={"ID":"de56150e-d7da-47da-8a86-1db61a99fd1d","Type":"ContainerDied","Data":"bc17a6dd94034892bd2229cdf97a43bcef06e86a98418d1bccccc561e4bb4e5a"}
Sep 11 12:13:30.570978 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:30.570951    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-fb3c56-m8zzg"
Sep 11 12:13:30.728189 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:30.728080    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/de56150e-d7da-47da-8a86-1db61a99fd1d-maas-api-service-ca\") pod \"de56150e-d7da-47da-8a86-1db61a99fd1d\" (UID: \"de56150e-d7da-47da-8a86-1db61a99fd1d\") "
Sep 11 12:13:30.728570 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:30.728543    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de56150e-d7da-47da-8a86-1db61a99fd1d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "de56150e-d7da-47da-8a86-1db61a99fd1d" (UID: "de56150e-d7da-47da-8a86-1db61a99fd1d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:13:30.829484 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:30.829444    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/de56150e-d7da-47da-8a86-1db61a99fd1d-maas-api-service-ca\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:13:31.428479 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:31.428449    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-fb3c56-m8zzg"
Sep 11 12:13:31.428479 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:31.428475    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-fb3c56-m8zzg" event={"ID":"de56150e-d7da-47da-8a86-1db61a99fd1d","Type":"ContainerDied","Data":"67d74a5ae339e9aedbb5052ce8b89c5cb06a8897cbbf07a2a0bda279e471f6bb"}
Sep 11 12:13:31.428710 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:31.428519    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="67d74a5ae339e9aedbb5052ce8b89c5cb06a8897cbbf07a2a0bda279e471f6bb"
Sep 11 12:13:37.359337 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:37.359290    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:13:42.633987 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:42.633781    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:13:42.664093 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:42.664053    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-fb3c56-d8c7bd578-l7nz4"]
Sep 11 12:13:42.664353 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:42.664307    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-fb3c56-d8c7bd578-l7nz4" podUID="48e1009c-d1cb-46a1-8a70-22efe471beb0" containerName="payload-pre-processing" containerID="cri-o://a4342186b2dfb69811cf24ae02f43fd6a9bf32dce030337bcd88910247b6c017" gracePeriod=30
Sep 11 12:13:42.735680 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:42.735634    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:13:42.838940 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:42.838824    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:13:42.840490 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:42.840337    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-fb3c56-m8zzg"]
Sep 11 12:13:42.842886 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:42.842859    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-fb3c56-m8zzg"]
Sep 11 12:13:42.927354 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:42.927330    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-fb3c56-d8c7bd578-l7nz4"
Sep 11 12:13:42.948989 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:42.948955    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:13:42.949459 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:42.949436    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/48e1009c-d1cb-46a1-8a70-22efe471beb0-plugins-config-volume\") pod \"48e1009c-d1cb-46a1-8a70-22efe471beb0\" (UID: \"48e1009c-d1cb-46a1-8a70-22efe471beb0\") "
Sep 11 12:13:42.949568 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:42.949521    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xh7rz\" (UniqueName: \"kubernetes.io/projected/48e1009c-d1cb-46a1-8a70-22efe471beb0-kube-api-access-xh7rz\") pod \"48e1009c-d1cb-46a1-8a70-22efe471beb0\" (UID: \"48e1009c-d1cb-46a1-8a70-22efe471beb0\") "
Sep 11 12:13:42.949868 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:42.949843    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48e1009c-d1cb-46a1-8a70-22efe471beb0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "48e1009c-d1cb-46a1-8a70-22efe471beb0" (UID: "48e1009c-d1cb-46a1-8a70-22efe471beb0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:13:42.951690 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:42.951661    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48e1009c-d1cb-46a1-8a70-22efe471beb0-kube-api-access-xh7rz" (OuterVolumeSpecName: "kube-api-access-xh7rz") pod "48e1009c-d1cb-46a1-8a70-22efe471beb0" (UID: "48e1009c-d1cb-46a1-8a70-22efe471beb0"). InnerVolumeSpecName "kube-api-access-xh7rz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:13:43.051095 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:43.051054    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/48e1009c-d1cb-46a1-8a70-22efe471beb0-plugins-config-volume\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:13:43.051095 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:43.051087    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xh7rz\" (UniqueName: \"kubernetes.io/projected/48e1009c-d1cb-46a1-8a70-22efe471beb0-kube-api-access-xh7rz\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:13:43.198114 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:43.198024    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:13:43.497880 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:43.497782    2579 generic.go:356] "Generic (PLEG): container finished" podID="48e1009c-d1cb-46a1-8a70-22efe471beb0" containerID="a4342186b2dfb69811cf24ae02f43fd6a9bf32dce030337bcd88910247b6c017" exitCode=0
Sep 11 12:13:43.497880 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:43.497828    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-fb3c56-d8c7bd578-l7nz4" event={"ID":"48e1009c-d1cb-46a1-8a70-22efe471beb0","Type":"ContainerDied","Data":"a4342186b2dfb69811cf24ae02f43fd6a9bf32dce030337bcd88910247b6c017"}
Sep 11 12:13:43.497880 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:43.497839    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-fb3c56-d8c7bd578-l7nz4"
Sep 11 12:13:43.498155 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:43.497882    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-fb3c56-d8c7bd578-l7nz4" event={"ID":"48e1009c-d1cb-46a1-8a70-22efe471beb0","Type":"ContainerDied","Data":"8ab47f28031981c899935e235d6e9aaf7ab348f9e0c76f73cd5cf1b96ed38e5b"}
Sep 11 12:13:43.498155 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:43.497908    2579 scope.go:117] "RemoveContainer" containerID="a4342186b2dfb69811cf24ae02f43fd6a9bf32dce030337bcd88910247b6c017"
Sep 11 12:13:43.507861 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:43.507848    2579 scope.go:117] "RemoveContainer" containerID="a4342186b2dfb69811cf24ae02f43fd6a9bf32dce030337bcd88910247b6c017"
Sep 11 12:13:43.508151 ip-10-0-142-145 kubenswrapper[2579]: E0911 12:13:43.508132    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4342186b2dfb69811cf24ae02f43fd6a9bf32dce030337bcd88910247b6c017\": container with ID starting with a4342186b2dfb69811cf24ae02f43fd6a9bf32dce030337bcd88910247b6c017 not found: ID does not exist" containerID="a4342186b2dfb69811cf24ae02f43fd6a9bf32dce030337bcd88910247b6c017"
Sep 11 12:13:43.508218 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:43.508165    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4342186b2dfb69811cf24ae02f43fd6a9bf32dce030337bcd88910247b6c017"} err="failed to get container status \"a4342186b2dfb69811cf24ae02f43fd6a9bf32dce030337bcd88910247b6c017\": rpc error: code = NotFound desc = could not find container \"a4342186b2dfb69811cf24ae02f43fd6a9bf32dce030337bcd88910247b6c017\": container with ID starting with a4342186b2dfb69811cf24ae02f43fd6a9bf32dce030337bcd88910247b6c017 not found: ID does not exist"
Sep 11 12:13:43.522422 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:43.522385    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-fb3c56-d8c7bd578-l7nz4"]
Sep 11 12:13:43.528071 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:43.528048    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-fb3c56-d8c7bd578-l7nz4"]
Sep 11 12:13:44.576611 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:44.576581    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48e1009c-d1cb-46a1-8a70-22efe471beb0" path="/var/lib/kubelet/pods/48e1009c-d1cb-46a1-8a70-22efe471beb0/volumes"
Sep 11 12:13:44.576991 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:44.576947    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de56150e-d7da-47da-8a86-1db61a99fd1d" path="/var/lib/kubelet/pods/de56150e-d7da-47da-8a86-1db61a99fd1d/volumes"
Sep 11 12:13:49.327752 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:49.327714    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:13:49.448560 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:49.448522    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:13:49.648456 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:49.648424    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg"]
Sep 11 12:13:49.648654 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:49.648634    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg" podUID="0a319527-8f12-4d0d-8609-3d15dc436d72" containerName="istio-proxy" containerID="cri-o://fd9819bc1b0f407c0a14e3297ff2b86242b06beb343189159b007384568a04cb" gracePeriod=30
Sep 11 12:13:49.694570 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:49.694529    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:13:49.931426 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:49.931312    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f4d189-jz82x"]
Sep 11 12:13:49.931787 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:49.931774    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48e1009c-d1cb-46a1-8a70-22efe471beb0" containerName="payload-pre-processing"
Sep 11 12:13:49.931834 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:49.931789    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="48e1009c-d1cb-46a1-8a70-22efe471beb0" containerName="payload-pre-processing"
Sep 11 12:13:49.931834 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:49.931798    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de56150e-d7da-47da-8a86-1db61a99fd1d" containerName="revoke-keys"
Sep 11 12:13:49.931834 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:49.931803    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="de56150e-d7da-47da-8a86-1db61a99fd1d" containerName="revoke-keys"
Sep 11 12:13:49.931949 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:49.931883    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="48e1009c-d1cb-46a1-8a70-22efe471beb0" containerName="payload-pre-processing"
Sep 11 12:13:49.931949 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:49.931896    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="de56150e-d7da-47da-8a86-1db61a99fd1d" containerName="revoke-keys"
Sep 11 12:13:49.935166 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:49.935148    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f4d189-jz82x"
Sep 11 12:13:49.938959 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:49.938933    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 12:13:49.938959 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:49.938947    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9xkpf\""
Sep 11 12:13:49.945672 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:49.945644    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f4d189-jz82x"]
Sep 11 12:13:50.020744 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:50.020708    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e9b9c1e-f687-4d1e-94f2-2d4cf2dd92dd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-f4d189-jz82x\" (UID: \"4e9b9c1e-f687-4d1e-94f2-2d4cf2dd92dd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f4d189-jz82x"
Sep 11 12:13:50.044423 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:50.044363    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:13:50.121339 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:50.121300    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e9b9c1e-f687-4d1e-94f2-2d4cf2dd92dd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-f4d189-jz82x\" (UID: \"4e9b9c1e-f687-4d1e-94f2-2d4cf2dd92dd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f4d189-jz82x"
Sep 11 12:13:50.121983 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:50.121963    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e9b9c1e-f687-4d1e-94f2-2d4cf2dd92dd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-f4d189-jz82x\" (UID: \"4e9b9c1e-f687-4d1e-94f2-2d4cf2dd92dd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f4d189-jz82x"
Sep 11 12:13:50.247152 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:50.247063    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f4d189-jz82x"
Sep 11 12:13:50.520236 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:50.520208    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f4d189-jz82x"]
Sep 11 12:13:50.522247 ip-10-0-142-145 kubenswrapper[2579]: W0911 12:13:50.522217    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e9b9c1e_f687_4d1e_94f2_2d4cf2dd92dd.slice/crio-aeb82f7c5cf9f30d12dec3419fb0502504eb4b20ebd68468dc7f1f974ffa33ae WatchSource:0}: Error finding container aeb82f7c5cf9f30d12dec3419fb0502504eb4b20ebd68468dc7f1f974ffa33ae: Status 404 returned error can't find the container with id aeb82f7c5cf9f30d12dec3419fb0502504eb4b20ebd68468dc7f1f974ffa33ae
Sep 11 12:13:50.547646 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:50.547617    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f4d189-jz82x" event={"ID":"4e9b9c1e-f687-4d1e-94f2-2d4cf2dd92dd","Type":"ContainerStarted","Data":"aeb82f7c5cf9f30d12dec3419fb0502504eb4b20ebd68468dc7f1f974ffa33ae"}
Sep 11 12:13:51.220817 ip-10-0-142-145 kubenswrapper[2579]: E0911 12:13:51.220754    2579 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e9b9c1e_f687_4d1e_94f2_2d4cf2dd92dd.slice/crio-7bb0b2039a2c952ee09d7b3e99b21063751739e19ebf4139614f5600ebc0a13e.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e9b9c1e_f687_4d1e_94f2_2d4cf2dd92dd.slice/crio-conmon-7bb0b2039a2c952ee09d7b3e99b21063751739e19ebf4139614f5600ebc0a13e.scope\": RecentStats: unable to find data in memory cache]"
Sep 11 12:13:51.557803 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:51.557709    2579 generic.go:356] "Generic (PLEG): container finished" podID="4e9b9c1e-f687-4d1e-94f2-2d4cf2dd92dd" containerID="7bb0b2039a2c952ee09d7b3e99b21063751739e19ebf4139614f5600ebc0a13e" exitCode=0
Sep 11 12:13:51.558215 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:51.557803    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f4d189-jz82x" event={"ID":"4e9b9c1e-f687-4d1e-94f2-2d4cf2dd92dd","Type":"ContainerDied","Data":"7bb0b2039a2c952ee09d7b3e99b21063751739e19ebf4139614f5600ebc0a13e"}
Sep 11 12:13:52.703664 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:52.703638    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f4d189-jz82x"
Sep 11 12:13:52.749161 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:52.749133    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e9b9c1e-f687-4d1e-94f2-2d4cf2dd92dd-maas-api-service-ca\") pod \"4e9b9c1e-f687-4d1e-94f2-2d4cf2dd92dd\" (UID: \"4e9b9c1e-f687-4d1e-94f2-2d4cf2dd92dd\") "
Sep 11 12:13:52.749605 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:52.749575    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e9b9c1e-f687-4d1e-94f2-2d4cf2dd92dd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4e9b9c1e-f687-4d1e-94f2-2d4cf2dd92dd" (UID: "4e9b9c1e-f687-4d1e-94f2-2d4cf2dd92dd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:13:52.850913 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:52.850876    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e9b9c1e-f687-4d1e-94f2-2d4cf2dd92dd-maas-api-service-ca\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:13:52.871710 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:52.871676    2579 patch_prober.go:28] interesting pod/e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 11 12:13:52.871865 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:52.871730    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg" podUID="0a319527-8f12-4d0d-8609-3d15dc436d72" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 12:13:53.234532 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:53.234443    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:13:53.570382 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:53.570308    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f4d189-jz82x"
Sep 11 12:13:53.570382 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:53.570345    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f4d189-jz82x" event={"ID":"4e9b9c1e-f687-4d1e-94f2-2d4cf2dd92dd","Type":"ContainerDied","Data":"aeb82f7c5cf9f30d12dec3419fb0502504eb4b20ebd68468dc7f1f974ffa33ae"}
Sep 11 12:13:53.570653 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:53.570386    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="aeb82f7c5cf9f30d12dec3419fb0502504eb4b20ebd68468dc7f1f974ffa33ae"
Sep 11 12:13:54.171053 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:54.171019    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:13:54.920454 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:54.920379    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg"
Sep 11 12:13:54.972253 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:54.972214    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a319527-8f12-4d0d-8609-3d15dc436d72-istio-envoy\") pod \"0a319527-8f12-4d0d-8609-3d15dc436d72\" (UID: \"0a319527-8f12-4d0d-8609-3d15dc436d72\") "
Sep 11 12:13:54.972537 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:54.972265    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a319527-8f12-4d0d-8609-3d15dc436d72-istiod-ca-cert\") pod \"0a319527-8f12-4d0d-8609-3d15dc436d72\" (UID: \"0a319527-8f12-4d0d-8609-3d15dc436d72\") "
Sep 11 12:13:54.972537 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:54.972312    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a319527-8f12-4d0d-8609-3d15dc436d72-workload-socket\") pod \"0a319527-8f12-4d0d-8609-3d15dc436d72\" (UID: \"0a319527-8f12-4d0d-8609-3d15dc436d72\") "
Sep 11 12:13:54.972537 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:54.972491    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a319527-8f12-4d0d-8609-3d15dc436d72-workload-certs\") pod \"0a319527-8f12-4d0d-8609-3d15dc436d72\" (UID: \"0a319527-8f12-4d0d-8609-3d15dc436d72\") "
Sep 11 12:13:54.972746 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:54.972559    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a319527-8f12-4d0d-8609-3d15dc436d72-istio-podinfo\") pod \"0a319527-8f12-4d0d-8609-3d15dc436d72\" (UID: \"0a319527-8f12-4d0d-8609-3d15dc436d72\") "
Sep 11 12:13:54.972746 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:54.972574    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a319527-8f12-4d0d-8609-3d15dc436d72-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0a319527-8f12-4d0d-8609-3d15dc436d72" (UID: "0a319527-8f12-4d0d-8609-3d15dc436d72"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:13:54.972746 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:54.972585    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a319527-8f12-4d0d-8609-3d15dc436d72-istio-token\") pod \"0a319527-8f12-4d0d-8609-3d15dc436d72\" (UID: \"0a319527-8f12-4d0d-8609-3d15dc436d72\") "
Sep 11 12:13:54.972746 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:54.972644    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a319527-8f12-4d0d-8609-3d15dc436d72-credential-socket\") pod \"0a319527-8f12-4d0d-8609-3d15dc436d72\" (UID: \"0a319527-8f12-4d0d-8609-3d15dc436d72\") "
Sep 11 12:13:54.972746 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:54.972742    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ftcm4\" (UniqueName: \"kubernetes.io/projected/0a319527-8f12-4d0d-8609-3d15dc436d72-kube-api-access-ftcm4\") pod \"0a319527-8f12-4d0d-8609-3d15dc436d72\" (UID: \"0a319527-8f12-4d0d-8609-3d15dc436d72\") "
Sep 11 12:13:54.973016 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:54.972744    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a319527-8f12-4d0d-8609-3d15dc436d72-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0a319527-8f12-4d0d-8609-3d15dc436d72" (UID: "0a319527-8f12-4d0d-8609-3d15dc436d72"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:13:54.973016 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:54.972772    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a319527-8f12-4d0d-8609-3d15dc436d72-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0a319527-8f12-4d0d-8609-3d15dc436d72" (UID: "0a319527-8f12-4d0d-8609-3d15dc436d72"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:13:54.973016 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:54.972816    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a319527-8f12-4d0d-8609-3d15dc436d72-istio-data\") pod \"0a319527-8f12-4d0d-8609-3d15dc436d72\" (UID: \"0a319527-8f12-4d0d-8609-3d15dc436d72\") "
Sep 11 12:13:54.973016 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:54.972981    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a319527-8f12-4d0d-8609-3d15dc436d72-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0a319527-8f12-4d0d-8609-3d15dc436d72" (UID: "0a319527-8f12-4d0d-8609-3d15dc436d72"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:13:54.973240 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:54.973158    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a319527-8f12-4d0d-8609-3d15dc436d72-istio-data" (OuterVolumeSpecName: "istio-data") pod "0a319527-8f12-4d0d-8609-3d15dc436d72" (UID: "0a319527-8f12-4d0d-8609-3d15dc436d72"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:13:54.973355 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:54.973333    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a319527-8f12-4d0d-8609-3d15dc436d72-istiod-ca-cert\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:13:54.973468 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:54.973359    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a319527-8f12-4d0d-8609-3d15dc436d72-workload-socket\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:13:54.973468 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:54.973375    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a319527-8f12-4d0d-8609-3d15dc436d72-workload-certs\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:13:54.973468 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:54.973420    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a319527-8f12-4d0d-8609-3d15dc436d72-credential-socket\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:13:54.973468 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:54.973437    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a319527-8f12-4d0d-8609-3d15dc436d72-istio-data\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:13:54.974709 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:54.974675    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a319527-8f12-4d0d-8609-3d15dc436d72-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0a319527-8f12-4d0d-8609-3d15dc436d72" (UID: "0a319527-8f12-4d0d-8609-3d15dc436d72"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:13:54.974947 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:54.974929    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a319527-8f12-4d0d-8609-3d15dc436d72-kube-api-access-ftcm4" (OuterVolumeSpecName: "kube-api-access-ftcm4") pod "0a319527-8f12-4d0d-8609-3d15dc436d72" (UID: "0a319527-8f12-4d0d-8609-3d15dc436d72"). InnerVolumeSpecName "kube-api-access-ftcm4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:13:54.975152 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:54.975136    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0a319527-8f12-4d0d-8609-3d15dc436d72-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0a319527-8f12-4d0d-8609-3d15dc436d72" (UID: "0a319527-8f12-4d0d-8609-3d15dc436d72"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 12:13:54.975376 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:54.975347    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a319527-8f12-4d0d-8609-3d15dc436d72-istio-token" (OuterVolumeSpecName: "istio-token") pod "0a319527-8f12-4d0d-8609-3d15dc436d72" (UID: "0a319527-8f12-4d0d-8609-3d15dc436d72"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:13:55.074878 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:55.074839    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a319527-8f12-4d0d-8609-3d15dc436d72-istio-podinfo\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:13:55.074878 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:55.074872    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a319527-8f12-4d0d-8609-3d15dc436d72-istio-token\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:13:55.074878 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:55.074884    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ftcm4\" (UniqueName: \"kubernetes.io/projected/0a319527-8f12-4d0d-8609-3d15dc436d72-kube-api-access-ftcm4\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:13:55.075129 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:55.074895    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a319527-8f12-4d0d-8609-3d15dc436d72-istio-envoy\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:13:55.584281 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:55.584245    2579 generic.go:356] "Generic (PLEG): container finished" podID="0a319527-8f12-4d0d-8609-3d15dc436d72" containerID="fd9819bc1b0f407c0a14e3297ff2b86242b06beb343189159b007384568a04cb" exitCode=0
Sep 11 12:13:55.584674 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:55.584303    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg"
Sep 11 12:13:55.584674 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:55.584333    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg" event={"ID":"0a319527-8f12-4d0d-8609-3d15dc436d72","Type":"ContainerDied","Data":"fd9819bc1b0f407c0a14e3297ff2b86242b06beb343189159b007384568a04cb"}
Sep 11 12:13:55.584674 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:55.584386    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg" event={"ID":"0a319527-8f12-4d0d-8609-3d15dc436d72","Type":"ContainerDied","Data":"a6f7381fdec585b81a69bf7befcc2e57343da315205ce6f6350519636875f6e7"}
Sep 11 12:13:55.584674 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:55.584418    2579 scope.go:117] "RemoveContainer" containerID="fd9819bc1b0f407c0a14e3297ff2b86242b06beb343189159b007384568a04cb"
Sep 11 12:13:55.600475 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:55.600454    2579 scope.go:117] "RemoveContainer" containerID="ca8d71d803faa1578dc8a0d50b6fc2d3a4aecf182c772509db0176885ea32957"
Sep 11 12:13:55.609353 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:55.609332    2579 scope.go:117] "RemoveContainer" containerID="fd9819bc1b0f407c0a14e3297ff2b86242b06beb343189159b007384568a04cb"
Sep 11 12:13:55.609694 ip-10-0-142-145 kubenswrapper[2579]: E0911 12:13:55.609671    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd9819bc1b0f407c0a14e3297ff2b86242b06beb343189159b007384568a04cb\": container with ID starting with fd9819bc1b0f407c0a14e3297ff2b86242b06beb343189159b007384568a04cb not found: ID does not exist" containerID="fd9819bc1b0f407c0a14e3297ff2b86242b06beb343189159b007384568a04cb"
Sep 11 12:13:55.609761 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:55.609705    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd9819bc1b0f407c0a14e3297ff2b86242b06beb343189159b007384568a04cb"} err="failed to get container status \"fd9819bc1b0f407c0a14e3297ff2b86242b06beb343189159b007384568a04cb\": rpc error: code = NotFound desc = could not find container \"fd9819bc1b0f407c0a14e3297ff2b86242b06beb343189159b007384568a04cb\": container with ID starting with fd9819bc1b0f407c0a14e3297ff2b86242b06beb343189159b007384568a04cb not found: ID does not exist"
Sep 11 12:13:55.609761 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:55.609724    2579 scope.go:117] "RemoveContainer" containerID="ca8d71d803faa1578dc8a0d50b6fc2d3a4aecf182c772509db0176885ea32957"
Sep 11 12:13:55.610083 ip-10-0-142-145 kubenswrapper[2579]: E0911 12:13:55.610056    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca8d71d803faa1578dc8a0d50b6fc2d3a4aecf182c772509db0176885ea32957\": container with ID starting with ca8d71d803faa1578dc8a0d50b6fc2d3a4aecf182c772509db0176885ea32957 not found: ID does not exist" containerID="ca8d71d803faa1578dc8a0d50b6fc2d3a4aecf182c772509db0176885ea32957"
Sep 11 12:13:55.610234 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:55.610091    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca8d71d803faa1578dc8a0d50b6fc2d3a4aecf182c772509db0176885ea32957"} err="failed to get container status \"ca8d71d803faa1578dc8a0d50b6fc2d3a4aecf182c772509db0176885ea32957\": rpc error: code = NotFound desc = could not find container \"ca8d71d803faa1578dc8a0d50b6fc2d3a4aecf182c772509db0176885ea32957\": container with ID starting with ca8d71d803faa1578dc8a0d50b6fc2d3a4aecf182c772509db0176885ea32957 not found: ID does not exist"
Sep 11 12:13:55.613364 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:55.613339    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg"]
Sep 11 12:13:55.617124 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:55.617102    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-fb3c56-openshift-default-56649fbddb-5g6mg"]
Sep 11 12:13:56.576881 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:13:56.576845    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a319527-8f12-4d0d-8609-3d15dc436d72" path="/var/lib/kubelet/pods/0a319527-8f12-4d0d-8609-3d15dc436d72/volumes"
Sep 11 12:14:00.066619 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:00.066588    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:01.010512 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:01.010478    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:02.285234 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:02.285201    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:02.384952 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:02.384925    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:02.699246 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:02.699210    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4"]
Sep 11 12:14:02.700665 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:02.700634    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a319527-8f12-4d0d-8609-3d15dc436d72" containerName="istio-proxy"
Sep 11 12:14:02.700821 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:02.700809    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a319527-8f12-4d0d-8609-3d15dc436d72" containerName="istio-proxy"
Sep 11 12:14:02.700942 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:02.700932    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e9b9c1e-f687-4d1e-94f2-2d4cf2dd92dd" containerName="revoke-keys"
Sep 11 12:14:02.701009 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:02.701001    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e9b9c1e-f687-4d1e-94f2-2d4cf2dd92dd" containerName="revoke-keys"
Sep 11 12:14:02.701295 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:02.701283    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e9b9c1e-f687-4d1e-94f2-2d4cf2dd92dd" containerName="revoke-keys"
Sep 11 12:14:02.701418 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:02.701384    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a319527-8f12-4d0d-8609-3d15dc436d72" containerName="istio-proxy"
Sep 11 12:14:02.701506 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:02.701497    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a319527-8f12-4d0d-8609-3d15dc436d72" containerName="istio-proxy"
Sep 11 12:14:02.707255 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:02.707234    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4"
Sep 11 12:14:02.710633 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:02.710611    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-bf9d5c-openshift-default-dockercfg-wqd9k\""
Sep 11 12:14:02.717167 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:02.717146    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4"]
Sep 11 12:14:02.726795 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:02.726772    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:02.832845 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:02.832805    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:02.846977 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:02.846948    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d361df08-ce57-4396-ab92-a869643ddb40-istio-envoy\") pod \"e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4\" (UID: \"d361df08-ce57-4396-ab92-a869643ddb40\") " pod="openshift-ingress/e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4"
Sep 11 12:14:02.847149 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:02.846985    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d361df08-ce57-4396-ab92-a869643ddb40-istio-data\") pod \"e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4\" (UID: \"d361df08-ce57-4396-ab92-a869643ddb40\") " pod="openshift-ingress/e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4"
Sep 11 12:14:02.847149 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:02.847007    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d361df08-ce57-4396-ab92-a869643ddb40-workload-certs\") pod \"e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4\" (UID: \"d361df08-ce57-4396-ab92-a869643ddb40\") " pod="openshift-ingress/e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4"
Sep 11 12:14:02.847149 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:02.847099    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d361df08-ce57-4396-ab92-a869643ddb40-workload-socket\") pod \"e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4\" (UID: \"d361df08-ce57-4396-ab92-a869643ddb40\") " pod="openshift-ingress/e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4"
Sep 11 12:14:02.847149 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:02.847126    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d361df08-ce57-4396-ab92-a869643ddb40-credential-socket\") pod \"e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4\" (UID: \"d361df08-ce57-4396-ab92-a869643ddb40\") " pod="openshift-ingress/e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4"
Sep 11 12:14:02.847313 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:02.847175    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ggq8\" (UniqueName: \"kubernetes.io/projected/d361df08-ce57-4396-ab92-a869643ddb40-kube-api-access-4ggq8\") pod \"e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4\" (UID: \"d361df08-ce57-4396-ab92-a869643ddb40\") " pod="openshift-ingress/e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4"
Sep 11 12:14:02.847313 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:02.847235    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d361df08-ce57-4396-ab92-a869643ddb40-istiod-ca-cert\") pod \"e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4\" (UID: \"d361df08-ce57-4396-ab92-a869643ddb40\") " pod="openshift-ingress/e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4"
Sep 11 12:14:02.847313 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:02.847267    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d361df08-ce57-4396-ab92-a869643ddb40-istio-token\") pod \"e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4\" (UID: \"d361df08-ce57-4396-ab92-a869643ddb40\") " pod="openshift-ingress/e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4"
Sep 11 12:14:02.847313 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:02.847287    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d361df08-ce57-4396-ab92-a869643ddb40-istio-podinfo\") pod \"e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4\" (UID: \"d361df08-ce57-4396-ab92-a869643ddb40\") " pod="openshift-ingress/e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4"
Sep 11 12:14:02.948014 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:02.947967    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d361df08-ce57-4396-ab92-a869643ddb40-istiod-ca-cert\") pod \"e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4\" (UID: \"d361df08-ce57-4396-ab92-a869643ddb40\") " pod="openshift-ingress/e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4"
Sep 11 12:14:02.948014 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:02.948016    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d361df08-ce57-4396-ab92-a869643ddb40-istio-token\") pod \"e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4\" (UID: \"d361df08-ce57-4396-ab92-a869643ddb40\") " pod="openshift-ingress/e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4"
Sep 11 12:14:02.948228 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:02.948131    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d361df08-ce57-4396-ab92-a869643ddb40-istio-podinfo\") pod \"e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4\" (UID: \"d361df08-ce57-4396-ab92-a869643ddb40\") " pod="openshift-ingress/e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4"
Sep 11 12:14:02.948270 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:02.948242    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d361df08-ce57-4396-ab92-a869643ddb40-istio-envoy\") pod \"e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4\" (UID: \"d361df08-ce57-4396-ab92-a869643ddb40\") " pod="openshift-ingress/e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4"
Sep 11 12:14:02.948305 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:02.948268    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d361df08-ce57-4396-ab92-a869643ddb40-istio-data\") pod \"e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4\" (UID: \"d361df08-ce57-4396-ab92-a869643ddb40\") " pod="openshift-ingress/e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4"
Sep 11 12:14:02.948305 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:02.948299    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d361df08-ce57-4396-ab92-a869643ddb40-workload-certs\") pod \"e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4\" (UID: \"d361df08-ce57-4396-ab92-a869643ddb40\") " pod="openshift-ingress/e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4"
Sep 11 12:14:02.948465 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:02.948443    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d361df08-ce57-4396-ab92-a869643ddb40-workload-socket\") pod \"e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4\" (UID: \"d361df08-ce57-4396-ab92-a869643ddb40\") " pod="openshift-ingress/e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4"
Sep 11 12:14:02.948570 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:02.948481    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d361df08-ce57-4396-ab92-a869643ddb40-credential-socket\") pod \"e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4\" (UID: \"d361df08-ce57-4396-ab92-a869643ddb40\") " pod="openshift-ingress/e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4"
Sep 11 12:14:02.948570 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:02.948563    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ggq8\" (UniqueName: \"kubernetes.io/projected/d361df08-ce57-4396-ab92-a869643ddb40-kube-api-access-4ggq8\") pod \"e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4\" (UID: \"d361df08-ce57-4396-ab92-a869643ddb40\") " pod="openshift-ingress/e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4"
Sep 11 12:14:02.948738 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:02.948716    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d361df08-ce57-4396-ab92-a869643ddb40-workload-certs\") pod \"e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4\" (UID: \"d361df08-ce57-4396-ab92-a869643ddb40\") " pod="openshift-ingress/e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4"
Sep 11 12:14:02.948804 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:02.948786    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d361df08-ce57-4396-ab92-a869643ddb40-istiod-ca-cert\") pod \"e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4\" (UID: \"d361df08-ce57-4396-ab92-a869643ddb40\") " pod="openshift-ingress/e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4"
Sep 11 12:14:02.948970 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:02.948948    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d361df08-ce57-4396-ab92-a869643ddb40-workload-socket\") pod \"e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4\" (UID: \"d361df08-ce57-4396-ab92-a869643ddb40\") " pod="openshift-ingress/e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4"
Sep 11 12:14:02.949040 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:02.949025    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d361df08-ce57-4396-ab92-a869643ddb40-istio-data\") pod \"e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4\" (UID: \"d361df08-ce57-4396-ab92-a869643ddb40\") " pod="openshift-ingress/e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4"
Sep 11 12:14:02.949084 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:02.949034    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d361df08-ce57-4396-ab92-a869643ddb40-credential-socket\") pod \"e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4\" (UID: \"d361df08-ce57-4396-ab92-a869643ddb40\") " pod="openshift-ingress/e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4"
Sep 11 12:14:02.950574 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:02.950517    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d361df08-ce57-4396-ab92-a869643ddb40-istio-envoy\") pod \"e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4\" (UID: \"d361df08-ce57-4396-ab92-a869643ddb40\") " pod="openshift-ingress/e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4"
Sep 11 12:14:02.950679 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:02.950662    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d361df08-ce57-4396-ab92-a869643ddb40-istio-podinfo\") pod \"e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4\" (UID: \"d361df08-ce57-4396-ab92-a869643ddb40\") " pod="openshift-ingress/e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4"
Sep 11 12:14:02.956335 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:02.956312    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ggq8\" (UniqueName: \"kubernetes.io/projected/d361df08-ce57-4396-ab92-a869643ddb40-kube-api-access-4ggq8\") pod \"e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4\" (UID: \"d361df08-ce57-4396-ab92-a869643ddb40\") " pod="openshift-ingress/e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4"
Sep 11 12:14:02.956469 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:02.956338    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d361df08-ce57-4396-ab92-a869643ddb40-istio-token\") pod \"e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4\" (UID: \"d361df08-ce57-4396-ab92-a869643ddb40\") " pod="openshift-ingress/e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4"
Sep 11 12:14:03.020565 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:03.020527    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4"
Sep 11 12:14:03.297901 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:03.297877    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4"]
Sep 11 12:14:03.299701 ip-10-0-142-145 kubenswrapper[2579]: W0911 12:14:03.299678    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd361df08_ce57_4396_ab92_a869643ddb40.slice/crio-018520f9d42950254397ad26a3956439414a8b82e05e185c55aae177bbcb2212 WatchSource:0}: Error finding container 018520f9d42950254397ad26a3956439414a8b82e05e185c55aae177bbcb2212: Status 404 returned error can't find the container with id 018520f9d42950254397ad26a3956439414a8b82e05e185c55aae177bbcb2212
Sep 11 12:14:03.302457 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:03.302426    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 12:14:03.302521 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:03.302487    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 12:14:03.302521 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:03.302517    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 12:14:03.631574 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:03.631541    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4" event={"ID":"d361df08-ce57-4396-ab92-a869643ddb40","Type":"ContainerStarted","Data":"79ede4f267ee1ca719c3113a9a26f78afe572addb1451dc20f70fb1722ed3dd0"}
Sep 11 12:14:03.631721 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:03.631597    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4" event={"ID":"d361df08-ce57-4396-ab92-a869643ddb40","Type":"ContainerStarted","Data":"018520f9d42950254397ad26a3956439414a8b82e05e185c55aae177bbcb2212"}
Sep 11 12:14:03.655372 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:03.655322    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4" podStartSLOduration=1.655306309 podStartE2EDuration="1.655306309s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:14:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:14:03.653604177 +0000 UTC m=+1547.551045935" watchObservedRunningTime="2026-09-11 12:14:03.655306309 +0000 UTC m=+1547.552748053"
Sep 11 12:14:04.021250 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:04.021158    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4"
Sep 11 12:14:04.026689 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:04.026661    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4"
Sep 11 12:14:04.637577 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:04.637544    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4"
Sep 11 12:14:04.638606 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:04.638587    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4"
Sep 11 12:14:05.557879 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:05.557849    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:05.654437 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:05.654354    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:05.758490 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:05.758461    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:05.811567 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:05.811487    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-f4d189-86bd5948b9-92cgn"]
Sep 11 12:14:05.811762 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:05.811736    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-f4d189-86bd5948b9-92cgn" podUID="6de1d37e-5930-45f4-8c73-b923133129a5" containerName="payload-processing" containerID="cri-o://1f3a9a9718ed4badbe9a55870d5195a7e4dff6475521ebd6f9208f0db6ac4eab" gracePeriod=30
Sep 11 12:14:05.889533 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:05.889497    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:06.023699 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:06.023659    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:06.079835 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:06.079775    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-f4d189-86bd5948b9-92cgn"
Sep 11 12:14:06.109561 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:06.109522    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:06.116570 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:06.116543    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f4d189-jz82x"]
Sep 11 12:14:06.119713 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:06.119689    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f4d189-jz82x"]
Sep 11 12:14:06.183860 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:06.183826    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w8l4r\" (UniqueName: \"kubernetes.io/projected/6de1d37e-5930-45f4-8c73-b923133129a5-kube-api-access-w8l4r\") pod \"6de1d37e-5930-45f4-8c73-b923133129a5\" (UID: \"6de1d37e-5930-45f4-8c73-b923133129a5\") "
Sep 11 12:14:06.184041 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:06.183921    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6de1d37e-5930-45f4-8c73-b923133129a5-plugins-config-volume\") pod \"6de1d37e-5930-45f4-8c73-b923133129a5\" (UID: \"6de1d37e-5930-45f4-8c73-b923133129a5\") "
Sep 11 12:14:06.184307 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:06.184283    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6de1d37e-5930-45f4-8c73-b923133129a5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6de1d37e-5930-45f4-8c73-b923133129a5" (UID: "6de1d37e-5930-45f4-8c73-b923133129a5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:14:06.186052 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:06.186029    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6de1d37e-5930-45f4-8c73-b923133129a5-kube-api-access-w8l4r" (OuterVolumeSpecName: "kube-api-access-w8l4r") pod "6de1d37e-5930-45f4-8c73-b923133129a5" (UID: "6de1d37e-5930-45f4-8c73-b923133129a5"). InnerVolumeSpecName "kube-api-access-w8l4r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:14:06.285615 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:06.285583    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w8l4r\" (UniqueName: \"kubernetes.io/projected/6de1d37e-5930-45f4-8c73-b923133129a5-kube-api-access-w8l4r\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:14:06.285615 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:06.285612    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6de1d37e-5930-45f4-8c73-b923133129a5-plugins-config-volume\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:14:06.433856 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:06.433818    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:06.534214 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:06.534170    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:06.577040 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:06.577004    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e9b9c1e-f687-4d1e-94f2-2d4cf2dd92dd" path="/var/lib/kubelet/pods/4e9b9c1e-f687-4d1e-94f2-2d4cf2dd92dd/volumes"
Sep 11 12:14:06.658607 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:06.658565    2579 generic.go:356] "Generic (PLEG): container finished" podID="6de1d37e-5930-45f4-8c73-b923133129a5" containerID="1f3a9a9718ed4badbe9a55870d5195a7e4dff6475521ebd6f9208f0db6ac4eab" exitCode=0
Sep 11 12:14:06.659087 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:06.658646    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-f4d189-86bd5948b9-92cgn"
Sep 11 12:14:06.659087 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:06.658646    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-f4d189-86bd5948b9-92cgn" event={"ID":"6de1d37e-5930-45f4-8c73-b923133129a5","Type":"ContainerDied","Data":"1f3a9a9718ed4badbe9a55870d5195a7e4dff6475521ebd6f9208f0db6ac4eab"}
Sep 11 12:14:06.659087 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:06.658764    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-f4d189-86bd5948b9-92cgn" event={"ID":"6de1d37e-5930-45f4-8c73-b923133129a5","Type":"ContainerDied","Data":"5bf7ddd84bc3345d04def17eb4067649874228097e9bc8ea347a0b7f65a85cd9"}
Sep 11 12:14:06.659087 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:06.658797    2579 scope.go:117] "RemoveContainer" containerID="1f3a9a9718ed4badbe9a55870d5195a7e4dff6475521ebd6f9208f0db6ac4eab"
Sep 11 12:14:06.674088 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:06.674060    2579 scope.go:117] "RemoveContainer" containerID="1f3a9a9718ed4badbe9a55870d5195a7e4dff6475521ebd6f9208f0db6ac4eab"
Sep 11 12:14:06.674513 ip-10-0-142-145 kubenswrapper[2579]: E0911 12:14:06.674489    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f3a9a9718ed4badbe9a55870d5195a7e4dff6475521ebd6f9208f0db6ac4eab\": container with ID starting with 1f3a9a9718ed4badbe9a55870d5195a7e4dff6475521ebd6f9208f0db6ac4eab not found: ID does not exist" containerID="1f3a9a9718ed4badbe9a55870d5195a7e4dff6475521ebd6f9208f0db6ac4eab"
Sep 11 12:14:06.674636 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:06.674535    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f3a9a9718ed4badbe9a55870d5195a7e4dff6475521ebd6f9208f0db6ac4eab"} err="failed to get container status \"1f3a9a9718ed4badbe9a55870d5195a7e4dff6475521ebd6f9208f0db6ac4eab\": rpc error: code = NotFound desc = could not find container \"1f3a9a9718ed4badbe9a55870d5195a7e4dff6475521ebd6f9208f0db6ac4eab\": container with ID starting with 1f3a9a9718ed4badbe9a55870d5195a7e4dff6475521ebd6f9208f0db6ac4eab not found: ID does not exist"
Sep 11 12:14:06.686942 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:06.686860    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-f4d189-86bd5948b9-92cgn"]
Sep 11 12:14:06.693752 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:06.693718    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-f4d189-86bd5948b9-92cgn"]
Sep 11 12:14:06.714406 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:06.714359    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:06.782150 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:06.782116    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:06.866655 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:06.866621    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:07.304451 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:07.304381    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:07.404167 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:07.404135    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:07.428808 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:07.428779    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-bf9d5c-5bc4c96fdb-fjvpp"]
Sep 11 12:14:07.429366 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:07.429331    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6de1d37e-5930-45f4-8c73-b923133129a5" containerName="payload-processing"
Sep 11 12:14:07.429366 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:07.429355    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="6de1d37e-5930-45f4-8c73-b923133129a5" containerName="payload-processing"
Sep 11 12:14:07.429594 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:07.429431    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a319527-8f12-4d0d-8609-3d15dc436d72" containerName="istio-proxy"
Sep 11 12:14:07.429594 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:07.429440    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a319527-8f12-4d0d-8609-3d15dc436d72" containerName="istio-proxy"
Sep 11 12:14:07.429594 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:07.429537    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="6de1d37e-5930-45f4-8c73-b923133129a5" containerName="payload-processing"
Sep 11 12:14:07.434075 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:07.434056    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-bf9d5c-5bc4c96fdb-fjvpp"
Sep 11 12:14:07.436770 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:07.436744    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-bf9d5c\""
Sep 11 12:14:07.436863 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:07.436811    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-bf9d5c-dockercfg-fm8sk\""
Sep 11 12:14:07.445209 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:07.445181    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-bf9d5c-5bc4c96fdb-fjvpp"]
Sep 11 12:14:07.599558 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:07.599520    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f18bfb8-3568-4039-8a1f-c4017c6beab7-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-bf9d5c-5bc4c96fdb-fjvpp\" (UID: \"4f18bfb8-3568-4039-8a1f-c4017c6beab7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-bf9d5c-5bc4c96fdb-fjvpp"
Sep 11 12:14:07.599758 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:07.599578    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84bnj\" (UniqueName: \"kubernetes.io/projected/4f18bfb8-3568-4039-8a1f-c4017c6beab7-kube-api-access-84bnj\") pod \"payload-pre-processing-e2e-ipp-a-bf9d5c-5bc4c96fdb-fjvpp\" (UID: \"4f18bfb8-3568-4039-8a1f-c4017c6beab7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-bf9d5c-5bc4c96fdb-fjvpp"
Sep 11 12:14:07.701459 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:07.701383    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f18bfb8-3568-4039-8a1f-c4017c6beab7-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-bf9d5c-5bc4c96fdb-fjvpp\" (UID: \"4f18bfb8-3568-4039-8a1f-c4017c6beab7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-bf9d5c-5bc4c96fdb-fjvpp"
Sep 11 12:14:07.701912 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:07.701503    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84bnj\" (UniqueName: \"kubernetes.io/projected/4f18bfb8-3568-4039-8a1f-c4017c6beab7-kube-api-access-84bnj\") pod \"payload-pre-processing-e2e-ipp-a-bf9d5c-5bc4c96fdb-fjvpp\" (UID: \"4f18bfb8-3568-4039-8a1f-c4017c6beab7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-bf9d5c-5bc4c96fdb-fjvpp"
Sep 11 12:14:07.702010 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:07.701991    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f18bfb8-3568-4039-8a1f-c4017c6beab7-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-bf9d5c-5bc4c96fdb-fjvpp\" (UID: \"4f18bfb8-3568-4039-8a1f-c4017c6beab7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-bf9d5c-5bc4c96fdb-fjvpp"
Sep 11 12:14:07.711021 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:07.710990    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-84bnj\" (UniqueName: \"kubernetes.io/projected/4f18bfb8-3568-4039-8a1f-c4017c6beab7-kube-api-access-84bnj\") pod \"payload-pre-processing-e2e-ipp-a-bf9d5c-5bc4c96fdb-fjvpp\" (UID: \"4f18bfb8-3568-4039-8a1f-c4017c6beab7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-bf9d5c-5bc4c96fdb-fjvpp"
Sep 11 12:14:07.745958 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:07.745926    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-bf9d5c-5bc4c96fdb-fjvpp"
Sep 11 12:14:08.018626 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:08.018575    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-bf9d5c-5bc4c96fdb-fjvpp"]
Sep 11 12:14:08.020927 ip-10-0-142-145 kubenswrapper[2579]: W0911 12:14:08.020897    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f18bfb8_3568_4039_8a1f_c4017c6beab7.slice/crio-33e55b575f0da5ccceb6fa1b29e31fd01322c96373ea03acc1b9597b00972d48 WatchSource:0}: Error finding container 33e55b575f0da5ccceb6fa1b29e31fd01322c96373ea03acc1b9597b00972d48: Status 404 returned error can't find the container with id 33e55b575f0da5ccceb6fa1b29e31fd01322c96373ea03acc1b9597b00972d48
Sep 11 12:14:08.577104 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:08.577069    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6de1d37e-5930-45f4-8c73-b923133129a5" path="/var/lib/kubelet/pods/6de1d37e-5930-45f4-8c73-b923133129a5/volumes"
Sep 11 12:14:08.673249 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:08.673217    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-bf9d5c-5bc4c96fdb-fjvpp" event={"ID":"4f18bfb8-3568-4039-8a1f-c4017c6beab7","Type":"ContainerStarted","Data":"2b02468ea580e1023de64d528ef0c937f3eaf5e424221ea34f113e43521eafb5"}
Sep 11 12:14:08.673461 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:08.673267    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-bf9d5c-5bc4c96fdb-fjvpp" event={"ID":"4f18bfb8-3568-4039-8a1f-c4017c6beab7","Type":"ContainerStarted","Data":"33e55b575f0da5ccceb6fa1b29e31fd01322c96373ea03acc1b9597b00972d48"}
Sep 11 12:14:08.673461 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:08.673318    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-bf9d5c-5bc4c96fdb-fjvpp"
Sep 11 12:14:08.693764 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:08.693714    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-bf9d5c-5bc4c96fdb-fjvpp" podStartSLOduration=1.69369887 podStartE2EDuration="1.69369887s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:14:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:14:08.691961424 +0000 UTC m=+1552.589403171" watchObservedRunningTime="2026-09-11 12:14:08.69369887 +0000 UTC m=+1552.591140645"
Sep 11 12:14:12.428081 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:12.428052    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:12.529333 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:12.529302    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:12.695324 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:12.695241    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:13.003628 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:13.003549    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-45352b-62lm6"]
Sep 11 12:14:13.008388 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:13.008371    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-45352b-62lm6"
Sep 11 12:14:13.011239 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:13.011202    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 12:14:13.011383 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:13.011274    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9xkpf\""
Sep 11 12:14:13.016164 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:13.016137    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-45352b-62lm6"]
Sep 11 12:14:13.121157 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:13.121117    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:13.152263 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:13.152220    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb8a7ce2-02c6-4971-9356-c88c061c1776-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-45352b-62lm6\" (UID: \"bb8a7ce2-02c6-4971-9356-c88c061c1776\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-45352b-62lm6"
Sep 11 12:14:13.253608 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:13.253563    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb8a7ce2-02c6-4971-9356-c88c061c1776-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-45352b-62lm6\" (UID: \"bb8a7ce2-02c6-4971-9356-c88c061c1776\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-45352b-62lm6"
Sep 11 12:14:13.254259 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:13.254204    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb8a7ce2-02c6-4971-9356-c88c061c1776-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-45352b-62lm6\" (UID: \"bb8a7ce2-02c6-4971-9356-c88c061c1776\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-45352b-62lm6"
Sep 11 12:14:13.320553 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:13.320509    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-45352b-62lm6"
Sep 11 12:14:13.597825 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:13.597795    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-45352b-62lm6"]
Sep 11 12:14:13.599890 ip-10-0-142-145 kubenswrapper[2579]: W0911 12:14:13.599863    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb8a7ce2_02c6_4971_9356_c88c061c1776.slice/crio-9955e88f91d0187ae9b8827c5e7d1f1c75675071299c6b98c5c529f09a79d22b WatchSource:0}: Error finding container 9955e88f91d0187ae9b8827c5e7d1f1c75675071299c6b98c5c529f09a79d22b: Status 404 returned error can't find the container with id 9955e88f91d0187ae9b8827c5e7d1f1c75675071299c6b98c5c529f09a79d22b
Sep 11 12:14:13.703415 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:13.703355    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-45352b-62lm6" event={"ID":"bb8a7ce2-02c6-4971-9356-c88c061c1776","Type":"ContainerStarted","Data":"9955e88f91d0187ae9b8827c5e7d1f1c75675071299c6b98c5c529f09a79d22b"}
Sep 11 12:14:13.824155 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:13.824122    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:14:13.987147 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:13.987044    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:14.712218 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:14.712181    2579 generic.go:356] "Generic (PLEG): container finished" podID="bb8a7ce2-02c6-4971-9356-c88c061c1776" containerID="6618ab43795d33de65f27e4a9b1ce6cf845114816930da8e89ca922808f9ea65" exitCode=0
Sep 11 12:14:14.712761 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:14.712238    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-45352b-62lm6" event={"ID":"bb8a7ce2-02c6-4971-9356-c88c061c1776","Type":"ContainerDied","Data":"6618ab43795d33de65f27e4a9b1ce6cf845114816930da8e89ca922808f9ea65"}
Sep 11 12:14:14.752447 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:14.752152    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:15.900421 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:15.888614    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-45352b-62lm6"
Sep 11 12:14:15.996954 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:15.995981    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb8a7ce2-02c6-4971-9356-c88c061c1776-maas-api-service-ca\") pod \"bb8a7ce2-02c6-4971-9356-c88c061c1776\" (UID: \"bb8a7ce2-02c6-4971-9356-c88c061c1776\") "
Sep 11 12:14:15.997363 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:15.997310    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb8a7ce2-02c6-4971-9356-c88c061c1776-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "bb8a7ce2-02c6-4971-9356-c88c061c1776" (UID: "bb8a7ce2-02c6-4971-9356-c88c061c1776"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:14:16.097421 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:16.097365    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb8a7ce2-02c6-4971-9356-c88c061c1776-maas-api-service-ca\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:14:16.723694 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:16.723661    2579 scope.go:117] "RemoveContainer" containerID="2e55580364b381c0e4decb71b7f77c9348f7699798e5c6d7f57ea468e52e206a"
Sep 11 12:14:16.727814 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:16.727783    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-45352b-62lm6"
Sep 11 12:14:16.727814 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:16.727804    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-45352b-62lm6" event={"ID":"bb8a7ce2-02c6-4971-9356-c88c061c1776","Type":"ContainerDied","Data":"9955e88f91d0187ae9b8827c5e7d1f1c75675071299c6b98c5c529f09a79d22b"}
Sep 11 12:14:16.728022 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:16.727848    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9955e88f91d0187ae9b8827c5e7d1f1c75675071299c6b98c5c529f09a79d22b"
Sep 11 12:14:19.681604 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:19.681565    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-bf9d5c-5bc4c96fdb-fjvpp"
Sep 11 12:14:23.544655 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:23.544626    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-45352b-5ddc97f6c8-tv8dc"]
Sep 11 12:14:23.545142 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:23.544824    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-45352b-5ddc97f6c8-tv8dc" podUID="7ef64f99-964e-4117-b678-2d0f7d025d66" containerName="payload-pre-processing" containerID="cri-o://43251685ac897dd4481322cefcf6d68ae56133c499021539dbe8aed9a23542e1" gracePeriod=30
Sep 11 12:14:23.770570 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:23.770536    2579 generic.go:356] "Generic (PLEG): container finished" podID="7ef64f99-964e-4117-b678-2d0f7d025d66" containerID="43251685ac897dd4481322cefcf6d68ae56133c499021539dbe8aed9a23542e1" exitCode=0
Sep 11 12:14:23.770796 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:23.770609    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-45352b-5ddc97f6c8-tv8dc" event={"ID":"7ef64f99-964e-4117-b678-2d0f7d025d66","Type":"ContainerDied","Data":"43251685ac897dd4481322cefcf6d68ae56133c499021539dbe8aed9a23542e1"}
Sep 11 12:14:23.772791 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:23.772763    2579 generic.go:356] "Generic (PLEG): container finished" podID="d361df08-ce57-4396-ab92-a869643ddb40" containerID="79ede4f267ee1ca719c3113a9a26f78afe572addb1451dc20f70fb1722ed3dd0" exitCode=0
Sep 11 12:14:23.772950 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:23.772843    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4" event={"ID":"d361df08-ce57-4396-ab92-a869643ddb40","Type":"ContainerDied","Data":"79ede4f267ee1ca719c3113a9a26f78afe572addb1451dc20f70fb1722ed3dd0"}
Sep 11 12:14:23.773335 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:23.773314    2579 scope.go:117] "RemoveContainer" containerID="79ede4f267ee1ca719c3113a9a26f78afe572addb1451dc20f70fb1722ed3dd0"
Sep 11 12:14:23.774625 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:23.774594    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 12:14:23.774719 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:23.774669    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 12:14:23.774719 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:23.774712    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 12:14:23.775339 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:23.775318    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-45352b-62lm6"]
Sep 11 12:14:23.779269 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:23.779246    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-45352b-62lm6"]
Sep 11 12:14:23.832717 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:23.832655    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-45352b-5ddc97f6c8-tv8dc"
Sep 11 12:14:23.880316 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:23.879050    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ef64f99-964e-4117-b678-2d0f7d025d66-plugins-config-volume\") pod \"7ef64f99-964e-4117-b678-2d0f7d025d66\" (UID: \"7ef64f99-964e-4117-b678-2d0f7d025d66\") "
Sep 11 12:14:23.880316 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:23.879212    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5h6b7\" (UniqueName: \"kubernetes.io/projected/7ef64f99-964e-4117-b678-2d0f7d025d66-kube-api-access-5h6b7\") pod \"7ef64f99-964e-4117-b678-2d0f7d025d66\" (UID: \"7ef64f99-964e-4117-b678-2d0f7d025d66\") "
Sep 11 12:14:23.880316 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:23.879671    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ef64f99-964e-4117-b678-2d0f7d025d66-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7ef64f99-964e-4117-b678-2d0f7d025d66" (UID: "7ef64f99-964e-4117-b678-2d0f7d025d66"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:14:23.883051 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:23.883004    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ef64f99-964e-4117-b678-2d0f7d025d66-kube-api-access-5h6b7" (OuterVolumeSpecName: "kube-api-access-5h6b7") pod "7ef64f99-964e-4117-b678-2d0f7d025d66" (UID: "7ef64f99-964e-4117-b678-2d0f7d025d66"). InnerVolumeSpecName "kube-api-access-5h6b7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:14:23.981061 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:23.980976    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ef64f99-964e-4117-b678-2d0f7d025d66-plugins-config-volume\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:14:23.981061 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:23.981012    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5h6b7\" (UniqueName: \"kubernetes.io/projected/7ef64f99-964e-4117-b678-2d0f7d025d66-kube-api-access-5h6b7\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:14:24.576968 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:24.576921    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb8a7ce2-02c6-4971-9356-c88c061c1776" path="/var/lib/kubelet/pods/bb8a7ce2-02c6-4971-9356-c88c061c1776/volumes"
Sep 11 12:14:24.585186 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:24.585138    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:24.685326 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:24.685293    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:24.782268 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:24.782224    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-45352b-5ddc97f6c8-tv8dc" event={"ID":"7ef64f99-964e-4117-b678-2d0f7d025d66","Type":"ContainerDied","Data":"a053f80a656996b13f7bdef87d38aa96d56f90efd0f0a208f8c09a08693a529a"}
Sep 11 12:14:24.782488 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:24.782306    2579 scope.go:117] "RemoveContainer" containerID="43251685ac897dd4481322cefcf6d68ae56133c499021539dbe8aed9a23542e1"
Sep 11 12:14:24.782488 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:24.782484    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-45352b-5ddc97f6c8-tv8dc"
Sep 11 12:14:24.788183 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:24.788147    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4" event={"ID":"d361df08-ce57-4396-ab92-a869643ddb40","Type":"ContainerStarted","Data":"c764a942ba0752066d3e2d5abdcbd085785f63a7bb4305412d18c2f0d5366d96"}
Sep 11 12:14:24.812291 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:24.812258    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-45352b-5ddc97f6c8-tv8dc"]
Sep 11 12:14:24.816819 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:24.816765    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-45352b-5ddc97f6c8-tv8dc"]
Sep 11 12:14:25.021489 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:25.021447    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4"
Sep 11 12:14:25.021719 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:25.021695    2579 patch_prober.go:28] interesting pod/e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Sep 11 12:14:25.021809 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:25.021755    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4" podUID="d361df08-ce57-4396-ab92-a869643ddb40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Sep 11 12:14:26.021031 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:26.020993    2579 patch_prober.go:28] interesting pod/e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Sep 11 12:14:26.021451 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:26.021061    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4" podUID="d361df08-ce57-4396-ab92-a869643ddb40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Sep 11 12:14:26.577188 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:26.577140    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ef64f99-964e-4117-b678-2d0f7d025d66" path="/var/lib/kubelet/pods/7ef64f99-964e-4117-b678-2d0f7d025d66/volumes"
Sep 11 12:14:27.021303 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:27.021255    2579 patch_prober.go:28] interesting pod/e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Sep 11 12:14:27.021841 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:27.021323    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4" podUID="d361df08-ce57-4396-ab92-a869643ddb40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Sep 11 12:14:27.722510 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:27.722473    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:28.020981 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:28.020885    2579 patch_prober.go:28] interesting pod/e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Sep 11 12:14:28.020981 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:28.020946    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4" podUID="d361df08-ce57-4396-ab92-a869643ddb40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Sep 11 12:14:28.578931 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:28.578892    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:14:29.021663 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:29.021627    2579 patch_prober.go:28] interesting pod/e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Sep 11 12:14:29.021842 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:29.021683    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4" podUID="d361df08-ce57-4396-ab92-a869643ddb40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Sep 11 12:14:30.213867 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:30.213830    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4"
Sep 11 12:14:30.214385 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:30.214299    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4"
Sep 11 12:14:30.215528 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:30.215503    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4"
Sep 11 12:14:30.495650 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:30.495564    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:30.661738 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:30.661704    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:30.762132 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:30.762021    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:30.894793 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:30.894750    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6"]
Sep 11 12:14:30.894996 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:30.894968    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6" podUID="650ba6c3-36f7-4f49-9cf7-230d2dcc7d77" containerName="istio-proxy" containerID="cri-o://5a6e84002e9a77ca84bf788bf6be258f970f9d55c83f8a8f22de04fa2a1555e4" gracePeriod=30
Sep 11 12:14:30.928248 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:30.928212    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:31.168739 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:31.168694    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:31.271850 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:31.271811    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f7851b-5qmzv"]
Sep 11 12:14:31.272306 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:31.272292    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb8a7ce2-02c6-4971-9356-c88c061c1776" containerName="revoke-keys"
Sep 11 12:14:31.272350 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:31.272308    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb8a7ce2-02c6-4971-9356-c88c061c1776" containerName="revoke-keys"
Sep 11 12:14:31.272350 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:31.272321    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ef64f99-964e-4117-b678-2d0f7d025d66" containerName="payload-pre-processing"
Sep 11 12:14:31.272350 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:31.272327    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ef64f99-964e-4117-b678-2d0f7d025d66" containerName="payload-pre-processing"
Sep 11 12:14:31.272485 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:31.272401    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ef64f99-964e-4117-b678-2d0f7d025d66" containerName="payload-pre-processing"
Sep 11 12:14:31.272485 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:31.272414    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb8a7ce2-02c6-4971-9356-c88c061c1776" containerName="revoke-keys"
Sep 11 12:14:31.275626 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:31.275609    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f7851b-5qmzv"
Sep 11 12:14:31.278424 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:31.278384    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 12:14:31.278424 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:31.278386    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9xkpf\""
Sep 11 12:14:31.285441 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:31.285412    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f7851b-5qmzv"]
Sep 11 12:14:31.363199 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:31.363165    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9bf573e8-f5df-4d9f-9015-d046efbe6c8d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-f7851b-5qmzv\" (UID: \"9bf573e8-f5df-4d9f-9015-d046efbe6c8d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f7851b-5qmzv"
Sep 11 12:14:31.383340 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:31.383296    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:31.463914 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:31.463817    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9bf573e8-f5df-4d9f-9015-d046efbe6c8d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-f7851b-5qmzv\" (UID: \"9bf573e8-f5df-4d9f-9015-d046efbe6c8d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f7851b-5qmzv"
Sep 11 12:14:31.464470 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:31.464450    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9bf573e8-f5df-4d9f-9015-d046efbe6c8d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-f7851b-5qmzv\" (UID: \"9bf573e8-f5df-4d9f-9015-d046efbe6c8d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f7851b-5qmzv"
Sep 11 12:14:31.482761 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:31.482722    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:31.587131 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:31.587090    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f7851b-5qmzv"
Sep 11 12:14:31.703174 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:31.703138    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:31.860164 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:31.860137    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f7851b-5qmzv"]
Sep 11 12:14:31.862042 ip-10-0-142-145 kubenswrapper[2579]: W0911 12:14:31.862010    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bf573e8_f5df_4d9f_9015_d046efbe6c8d.slice/crio-d1fa644c63c94b7d52aa4e04125375c786f7882acf4df0268065394549e5bf89 WatchSource:0}: Error finding container d1fa644c63c94b7d52aa4e04125375c786f7882acf4df0268065394549e5bf89: Status 404 returned error can't find the container with id d1fa644c63c94b7d52aa4e04125375c786f7882acf4df0268065394549e5bf89
Sep 11 12:14:32.841509 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:32.841416    2579 generic.go:356] "Generic (PLEG): container finished" podID="9bf573e8-f5df-4d9f-9015-d046efbe6c8d" containerID="15a7870ebdfe2d67b1a132a8f33dbfa40dd440b9abb730dd9021881fd8b91a89" exitCode=0
Sep 11 12:14:32.841509 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:32.841461    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f7851b-5qmzv" event={"ID":"9bf573e8-f5df-4d9f-9015-d046efbe6c8d","Type":"ContainerDied","Data":"15a7870ebdfe2d67b1a132a8f33dbfa40dd440b9abb730dd9021881fd8b91a89"}
Sep 11 12:14:32.841905 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:32.841508    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f7851b-5qmzv" event={"ID":"9bf573e8-f5df-4d9f-9015-d046efbe6c8d","Type":"ContainerStarted","Data":"d1fa644c63c94b7d52aa4e04125375c786f7882acf4df0268065394549e5bf89"}
Sep 11 12:14:34.001815 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:34.001791    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f7851b-5qmzv"
Sep 11 12:14:34.094148 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:34.094104    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9bf573e8-f5df-4d9f-9015-d046efbe6c8d-maas-api-service-ca\") pod \"9bf573e8-f5df-4d9f-9015-d046efbe6c8d\" (UID: \"9bf573e8-f5df-4d9f-9015-d046efbe6c8d\") "
Sep 11 12:14:34.094522 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:34.094497    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9bf573e8-f5df-4d9f-9015-d046efbe6c8d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9bf573e8-f5df-4d9f-9015-d046efbe6c8d" (UID: "9bf573e8-f5df-4d9f-9015-d046efbe6c8d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:14:34.195316 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:34.195229    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9bf573e8-f5df-4d9f-9015-d046efbe6c8d-maas-api-service-ca\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:14:34.853316 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:34.853282    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f7851b-5qmzv" event={"ID":"9bf573e8-f5df-4d9f-9015-d046efbe6c8d","Type":"ContainerDied","Data":"d1fa644c63c94b7d52aa4e04125375c786f7882acf4df0268065394549e5bf89"}
Sep 11 12:14:34.853517 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:34.853321    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f7851b-5qmzv"
Sep 11 12:14:34.853517 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:34.853358    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d1fa644c63c94b7d52aa4e04125375c786f7882acf4df0268065394549e5bf89"
Sep 11 12:14:34.937939 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:34.937902    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:35.086461 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:35.086416    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:36.105754 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:36.105718    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:36.160863 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:36.160797    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6"
Sep 11 12:14:36.188678 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:36.188642    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:36.317243 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:36.317203    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/650ba6c3-36f7-4f49-9cf7-230d2dcc7d77-istio-envoy\") pod \"650ba6c3-36f7-4f49-9cf7-230d2dcc7d77\" (UID: \"650ba6c3-36f7-4f49-9cf7-230d2dcc7d77\") "
Sep 11 12:14:36.317458 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:36.317255    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/650ba6c3-36f7-4f49-9cf7-230d2dcc7d77-credential-socket\") pod \"650ba6c3-36f7-4f49-9cf7-230d2dcc7d77\" (UID: \"650ba6c3-36f7-4f49-9cf7-230d2dcc7d77\") "
Sep 11 12:14:36.317458 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:36.317292    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-flp7t\" (UniqueName: \"kubernetes.io/projected/650ba6c3-36f7-4f49-9cf7-230d2dcc7d77-kube-api-access-flp7t\") pod \"650ba6c3-36f7-4f49-9cf7-230d2dcc7d77\" (UID: \"650ba6c3-36f7-4f49-9cf7-230d2dcc7d77\") "
Sep 11 12:14:36.317458 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:36.317322    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/650ba6c3-36f7-4f49-9cf7-230d2dcc7d77-istio-podinfo\") pod \"650ba6c3-36f7-4f49-9cf7-230d2dcc7d77\" (UID: \"650ba6c3-36f7-4f49-9cf7-230d2dcc7d77\") "
Sep 11 12:14:36.317458 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:36.317348    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/650ba6c3-36f7-4f49-9cf7-230d2dcc7d77-istio-data\") pod \"650ba6c3-36f7-4f49-9cf7-230d2dcc7d77\" (UID: \"650ba6c3-36f7-4f49-9cf7-230d2dcc7d77\") "
Sep 11 12:14:36.317458 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:36.317419    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/650ba6c3-36f7-4f49-9cf7-230d2dcc7d77-workload-socket\") pod \"650ba6c3-36f7-4f49-9cf7-230d2dcc7d77\" (UID: \"650ba6c3-36f7-4f49-9cf7-230d2dcc7d77\") "
Sep 11 12:14:36.317458 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:36.317450    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/650ba6c3-36f7-4f49-9cf7-230d2dcc7d77-workload-certs\") pod \"650ba6c3-36f7-4f49-9cf7-230d2dcc7d77\" (UID: \"650ba6c3-36f7-4f49-9cf7-230d2dcc7d77\") "
Sep 11 12:14:36.317805 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:36.317504    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/650ba6c3-36f7-4f49-9cf7-230d2dcc7d77-istiod-ca-cert\") pod \"650ba6c3-36f7-4f49-9cf7-230d2dcc7d77\" (UID: \"650ba6c3-36f7-4f49-9cf7-230d2dcc7d77\") "
Sep 11 12:14:36.317805 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:36.317571    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/650ba6c3-36f7-4f49-9cf7-230d2dcc7d77-istio-token\") pod \"650ba6c3-36f7-4f49-9cf7-230d2dcc7d77\" (UID: \"650ba6c3-36f7-4f49-9cf7-230d2dcc7d77\") "
Sep 11 12:14:36.317805 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:36.317596    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/650ba6c3-36f7-4f49-9cf7-230d2dcc7d77-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "650ba6c3-36f7-4f49-9cf7-230d2dcc7d77" (UID: "650ba6c3-36f7-4f49-9cf7-230d2dcc7d77"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:14:36.317805 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:36.317698    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/650ba6c3-36f7-4f49-9cf7-230d2dcc7d77-istio-data" (OuterVolumeSpecName: "istio-data") pod "650ba6c3-36f7-4f49-9cf7-230d2dcc7d77" (UID: "650ba6c3-36f7-4f49-9cf7-230d2dcc7d77"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:14:36.318037 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:36.317827    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/650ba6c3-36f7-4f49-9cf7-230d2dcc7d77-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "650ba6c3-36f7-4f49-9cf7-230d2dcc7d77" (UID: "650ba6c3-36f7-4f49-9cf7-230d2dcc7d77"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:14:36.318037 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:36.317940    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/650ba6c3-36f7-4f49-9cf7-230d2dcc7d77-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "650ba6c3-36f7-4f49-9cf7-230d2dcc7d77" (UID: "650ba6c3-36f7-4f49-9cf7-230d2dcc7d77"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:14:36.318037 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:36.318028    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/650ba6c3-36f7-4f49-9cf7-230d2dcc7d77-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "650ba6c3-36f7-4f49-9cf7-230d2dcc7d77" (UID: "650ba6c3-36f7-4f49-9cf7-230d2dcc7d77"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:14:36.318216 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:36.318039    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/650ba6c3-36f7-4f49-9cf7-230d2dcc7d77-istio-data\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:14:36.318216 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:36.318070    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/650ba6c3-36f7-4f49-9cf7-230d2dcc7d77-workload-socket\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:14:36.318216 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:36.318086    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/650ba6c3-36f7-4f49-9cf7-230d2dcc7d77-workload-certs\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:14:36.318216 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:36.318099    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/650ba6c3-36f7-4f49-9cf7-230d2dcc7d77-credential-socket\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:14:36.319684 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:36.319655    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/650ba6c3-36f7-4f49-9cf7-230d2dcc7d77-kube-api-access-flp7t" (OuterVolumeSpecName: "kube-api-access-flp7t") pod "650ba6c3-36f7-4f49-9cf7-230d2dcc7d77" (UID: "650ba6c3-36f7-4f49-9cf7-230d2dcc7d77"). InnerVolumeSpecName "kube-api-access-flp7t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:14:36.319798 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:36.319681    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/650ba6c3-36f7-4f49-9cf7-230d2dcc7d77-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "650ba6c3-36f7-4f49-9cf7-230d2dcc7d77" (UID: "650ba6c3-36f7-4f49-9cf7-230d2dcc7d77"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:14:36.319847 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:36.319784    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/650ba6c3-36f7-4f49-9cf7-230d2dcc7d77-istio-token" (OuterVolumeSpecName: "istio-token") pod "650ba6c3-36f7-4f49-9cf7-230d2dcc7d77" (UID: "650ba6c3-36f7-4f49-9cf7-230d2dcc7d77"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:14:36.319893 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:36.319839    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/650ba6c3-36f7-4f49-9cf7-230d2dcc7d77-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "650ba6c3-36f7-4f49-9cf7-230d2dcc7d77" (UID: "650ba6c3-36f7-4f49-9cf7-230d2dcc7d77"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 12:14:36.418648 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:36.418568    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/650ba6c3-36f7-4f49-9cf7-230d2dcc7d77-istio-envoy\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:14:36.418648 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:36.418595    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-flp7t\" (UniqueName: \"kubernetes.io/projected/650ba6c3-36f7-4f49-9cf7-230d2dcc7d77-kube-api-access-flp7t\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:14:36.418648 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:36.418605    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/650ba6c3-36f7-4f49-9cf7-230d2dcc7d77-istio-podinfo\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:14:36.418648 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:36.418614    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/650ba6c3-36f7-4f49-9cf7-230d2dcc7d77-istiod-ca-cert\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:14:36.418648 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:36.418621    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/650ba6c3-36f7-4f49-9cf7-230d2dcc7d77-istio-token\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:14:36.623508 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:36.623469    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:36.751916 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:36.751832    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:36.844125 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:36.844091    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:36.852775 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:36.852740    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-986cf2-57cd48794b-t8qcx"]
Sep 11 12:14:36.853285 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:36.853265    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="650ba6c3-36f7-4f49-9cf7-230d2dcc7d77" containerName="istio-proxy"
Sep 11 12:14:36.853384 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:36.853288    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="650ba6c3-36f7-4f49-9cf7-230d2dcc7d77" containerName="istio-proxy"
Sep 11 12:14:36.853384 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:36.853304    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bf573e8-f5df-4d9f-9015-d046efbe6c8d" containerName="revoke-keys"
Sep 11 12:14:36.853384 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:36.853313    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bf573e8-f5df-4d9f-9015-d046efbe6c8d" containerName="revoke-keys"
Sep 11 12:14:36.853572 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:36.853458    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="650ba6c3-36f7-4f49-9cf7-230d2dcc7d77" containerName="istio-proxy"
Sep 11 12:14:36.853572 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:36.853471    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="9bf573e8-f5df-4d9f-9015-d046efbe6c8d" containerName="revoke-keys"
Sep 11 12:14:36.853572 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:36.853484    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="650ba6c3-36f7-4f49-9cf7-230d2dcc7d77" containerName="istio-proxy"
Sep 11 12:14:36.856105 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:36.856082    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-986cf2-57cd48794b-t8qcx"
Sep 11 12:14:36.858895 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:36.858878    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-986cf2\""
Sep 11 12:14:36.867856 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:36.867823    2579 generic.go:356] "Generic (PLEG): container finished" podID="650ba6c3-36f7-4f49-9cf7-230d2dcc7d77" containerID="5a6e84002e9a77ca84bf788bf6be258f970f9d55c83f8a8f22de04fa2a1555e4" exitCode=0
Sep 11 12:14:36.867994 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:36.867886    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6"
Sep 11 12:14:36.867994 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:36.867906    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6" event={"ID":"650ba6c3-36f7-4f49-9cf7-230d2dcc7d77","Type":"ContainerDied","Data":"5a6e84002e9a77ca84bf788bf6be258f970f9d55c83f8a8f22de04fa2a1555e4"}
Sep 11 12:14:36.867994 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:36.867957    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6" event={"ID":"650ba6c3-36f7-4f49-9cf7-230d2dcc7d77","Type":"ContainerDied","Data":"bf512af858cbf586a5738eb610a96597957b43f9003ddc7c20c45083192f34be"}
Sep 11 12:14:36.867994 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:36.867981    2579 scope.go:117] "RemoveContainer" containerID="5a6e84002e9a77ca84bf788bf6be258f970f9d55c83f8a8f22de04fa2a1555e4"
Sep 11 12:14:36.868845 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:36.868565    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-986cf2-57cd48794b-t8qcx"]
Sep 11 12:14:36.871829 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:36.871803    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-986cf2-65674c9d59-q6f47"]
Sep 11 12:14:36.872346 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:36.872328    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="650ba6c3-36f7-4f49-9cf7-230d2dcc7d77" containerName="istio-proxy"
Sep 11 12:14:36.872346 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:36.872347    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="650ba6c3-36f7-4f49-9cf7-230d2dcc7d77" containerName="istio-proxy"
Sep 11 12:14:36.874822 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:36.874782    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-986cf2-65674c9d59-q6f47"
Sep 11 12:14:36.877795 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:36.877772    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-986cf2-dockercfg-hmpq8\""
Sep 11 12:14:36.879648 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:36.879609    2579 scope.go:117] "RemoveContainer" containerID="fd31c201c55293320d1f67a85243080b52b5a4c08bf2561f706257131f72c2f3"
Sep 11 12:14:36.890902 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:36.890878    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-986cf2-65674c9d59-q6f47"]
Sep 11 12:14:36.894304 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:36.894285    2579 scope.go:117] "RemoveContainer" containerID="5a6e84002e9a77ca84bf788bf6be258f970f9d55c83f8a8f22de04fa2a1555e4"
Sep 11 12:14:36.894639 ip-10-0-142-145 kubenswrapper[2579]: E0911 12:14:36.894618    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a6e84002e9a77ca84bf788bf6be258f970f9d55c83f8a8f22de04fa2a1555e4\": container with ID starting with 5a6e84002e9a77ca84bf788bf6be258f970f9d55c83f8a8f22de04fa2a1555e4 not found: ID does not exist" containerID="5a6e84002e9a77ca84bf788bf6be258f970f9d55c83f8a8f22de04fa2a1555e4"
Sep 11 12:14:36.894730 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:36.894646    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a6e84002e9a77ca84bf788bf6be258f970f9d55c83f8a8f22de04fa2a1555e4"} err="failed to get container status \"5a6e84002e9a77ca84bf788bf6be258f970f9d55c83f8a8f22de04fa2a1555e4\": rpc error: code = NotFound desc = could not find container \"5a6e84002e9a77ca84bf788bf6be258f970f9d55c83f8a8f22de04fa2a1555e4\": container with ID starting with 5a6e84002e9a77ca84bf788bf6be258f970f9d55c83f8a8f22de04fa2a1555e4 not found: ID does not exist"
Sep 11 12:14:36.894730 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:36.894664    2579 scope.go:117] "RemoveContainer" containerID="fd31c201c55293320d1f67a85243080b52b5a4c08bf2561f706257131f72c2f3"
Sep 11 12:14:36.894951 ip-10-0-142-145 kubenswrapper[2579]: E0911 12:14:36.894926    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd31c201c55293320d1f67a85243080b52b5a4c08bf2561f706257131f72c2f3\": container with ID starting with fd31c201c55293320d1f67a85243080b52b5a4c08bf2561f706257131f72c2f3 not found: ID does not exist" containerID="fd31c201c55293320d1f67a85243080b52b5a4c08bf2561f706257131f72c2f3"
Sep 11 12:14:36.895000 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:36.894955    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd31c201c55293320d1f67a85243080b52b5a4c08bf2561f706257131f72c2f3"} err="failed to get container status \"fd31c201c55293320d1f67a85243080b52b5a4c08bf2561f706257131f72c2f3\": rpc error: code = NotFound desc = could not find container \"fd31c201c55293320d1f67a85243080b52b5a4c08bf2561f706257131f72c2f3\": container with ID starting with fd31c201c55293320d1f67a85243080b52b5a4c08bf2561f706257131f72c2f3 not found: ID does not exist"
Sep 11 12:14:36.920925 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:36.920889    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6"]
Sep 11 12:14:36.922354 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:36.922326    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lwsw\" (UniqueName: \"kubernetes.io/projected/561d8ff4-6ec4-410a-bfae-409e687b916b-kube-api-access-4lwsw\") pod \"payload-pre-processing-e2e-ipp-b-986cf2-57cd48794b-t8qcx\" (UID: \"561d8ff4-6ec4-410a-bfae-409e687b916b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-986cf2-57cd48794b-t8qcx"
Sep 11 12:14:36.922548 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:36.922424    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/561d8ff4-6ec4-410a-bfae-409e687b916b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-986cf2-57cd48794b-t8qcx\" (UID: \"561d8ff4-6ec4-410a-bfae-409e687b916b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-986cf2-57cd48794b-t8qcx"
Sep 11 12:14:36.924404 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:36.924363    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-45352b-openshift-default-58896f9cb8-ljlp6"]
Sep 11 12:14:37.023047 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:37.022958    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/561d8ff4-6ec4-410a-bfae-409e687b916b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-986cf2-57cd48794b-t8qcx\" (UID: \"561d8ff4-6ec4-410a-bfae-409e687b916b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-986cf2-57cd48794b-t8qcx"
Sep 11 12:14:37.023047 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:37.023006    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4e15f705-fc9a-4af7-b05a-5c359c81ac87-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-986cf2-65674c9d59-q6f47\" (UID: \"4e15f705-fc9a-4af7-b05a-5c359c81ac87\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-986cf2-65674c9d59-q6f47"
Sep 11 12:14:37.023275 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:37.023087    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lwsw\" (UniqueName: \"kubernetes.io/projected/561d8ff4-6ec4-410a-bfae-409e687b916b-kube-api-access-4lwsw\") pod \"payload-pre-processing-e2e-ipp-b-986cf2-57cd48794b-t8qcx\" (UID: \"561d8ff4-6ec4-410a-bfae-409e687b916b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-986cf2-57cd48794b-t8qcx"
Sep 11 12:14:37.023275 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:37.023120    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tpx2f\" (UniqueName: \"kubernetes.io/projected/4e15f705-fc9a-4af7-b05a-5c359c81ac87-kube-api-access-tpx2f\") pod \"payload-processing-e2e-ipp-b-986cf2-65674c9d59-q6f47\" (UID: \"4e15f705-fc9a-4af7-b05a-5c359c81ac87\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-986cf2-65674c9d59-q6f47"
Sep 11 12:14:37.023703 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:37.023681    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/561d8ff4-6ec4-410a-bfae-409e687b916b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-986cf2-57cd48794b-t8qcx\" (UID: \"561d8ff4-6ec4-410a-bfae-409e687b916b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-986cf2-57cd48794b-t8qcx"
Sep 11 12:14:37.031645 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:37.031624    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lwsw\" (UniqueName: \"kubernetes.io/projected/561d8ff4-6ec4-410a-bfae-409e687b916b-kube-api-access-4lwsw\") pod \"payload-pre-processing-e2e-ipp-b-986cf2-57cd48794b-t8qcx\" (UID: \"561d8ff4-6ec4-410a-bfae-409e687b916b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-986cf2-57cd48794b-t8qcx"
Sep 11 12:14:37.124344 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:37.124307    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpx2f\" (UniqueName: \"kubernetes.io/projected/4e15f705-fc9a-4af7-b05a-5c359c81ac87-kube-api-access-tpx2f\") pod \"payload-processing-e2e-ipp-b-986cf2-65674c9d59-q6f47\" (UID: \"4e15f705-fc9a-4af7-b05a-5c359c81ac87\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-986cf2-65674c9d59-q6f47"
Sep 11 12:14:37.124861 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:37.124387    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4e15f705-fc9a-4af7-b05a-5c359c81ac87-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-986cf2-65674c9d59-q6f47\" (UID: \"4e15f705-fc9a-4af7-b05a-5c359c81ac87\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-986cf2-65674c9d59-q6f47"
Sep 11 12:14:37.125204 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:37.125179    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4e15f705-fc9a-4af7-b05a-5c359c81ac87-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-986cf2-65674c9d59-q6f47\" (UID: \"4e15f705-fc9a-4af7-b05a-5c359c81ac87\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-986cf2-65674c9d59-q6f47"
Sep 11 12:14:37.134461 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:37.134433    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tpx2f\" (UniqueName: \"kubernetes.io/projected/4e15f705-fc9a-4af7-b05a-5c359c81ac87-kube-api-access-tpx2f\") pod \"payload-processing-e2e-ipp-b-986cf2-65674c9d59-q6f47\" (UID: \"4e15f705-fc9a-4af7-b05a-5c359c81ac87\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-986cf2-65674c9d59-q6f47"
Sep 11 12:14:37.168485 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:37.168452    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-986cf2-57cd48794b-t8qcx"
Sep 11 12:14:37.190257 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:37.190217    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-986cf2-65674c9d59-q6f47"
Sep 11 12:14:37.449408 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:37.449370    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-986cf2-57cd48794b-t8qcx"]
Sep 11 12:14:37.451074 ip-10-0-142-145 kubenswrapper[2579]: W0911 12:14:37.451047    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod561d8ff4_6ec4_410a_bfae_409e687b916b.slice/crio-b4c58dfcf92c7664c77d11b5bfd82d0666f99c878a12a31ba590f5b642068290 WatchSource:0}: Error finding container b4c58dfcf92c7664c77d11b5bfd82d0666f99c878a12a31ba590f5b642068290: Status 404 returned error can't find the container with id b4c58dfcf92c7664c77d11b5bfd82d0666f99c878a12a31ba590f5b642068290
Sep 11 12:14:37.475740 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:37.475716    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-986cf2-65674c9d59-q6f47"]
Sep 11 12:14:37.479771 ip-10-0-142-145 kubenswrapper[2579]: W0911 12:14:37.479745    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e15f705_fc9a_4af7_b05a_5c359c81ac87.slice/crio-ef854a1046d995164d38c131e0183cc7cfee239eefc2b4afc607bc4787c6393d WatchSource:0}: Error finding container ef854a1046d995164d38c131e0183cc7cfee239eefc2b4afc607bc4787c6393d: Status 404 returned error can't find the container with id ef854a1046d995164d38c131e0183cc7cfee239eefc2b4afc607bc4787c6393d
Sep 11 12:14:37.882208 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:37.882155    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-986cf2-65674c9d59-q6f47" event={"ID":"4e15f705-fc9a-4af7-b05a-5c359c81ac87","Type":"ContainerStarted","Data":"35081373072a02f34a27321f26afdfc31e9b9df7980d39c01cd6f85568bd6f6c"}
Sep 11 12:14:37.882441 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:37.882239    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-986cf2-65674c9d59-q6f47"
Sep 11 12:14:37.882441 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:37.882265    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-986cf2-65674c9d59-q6f47" event={"ID":"4e15f705-fc9a-4af7-b05a-5c359c81ac87","Type":"ContainerStarted","Data":"ef854a1046d995164d38c131e0183cc7cfee239eefc2b4afc607bc4787c6393d"}
Sep 11 12:14:37.885220 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:37.885186    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-986cf2-57cd48794b-t8qcx" event={"ID":"561d8ff4-6ec4-410a-bfae-409e687b916b","Type":"ContainerStarted","Data":"6dfa2af2f57b6e445b20d78e7b7cc9984222a72e1572d96d4a7935d42b145239"}
Sep 11 12:14:37.885367 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:37.885240    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-986cf2-57cd48794b-t8qcx" event={"ID":"561d8ff4-6ec4-410a-bfae-409e687b916b","Type":"ContainerStarted","Data":"b4c58dfcf92c7664c77d11b5bfd82d0666f99c878a12a31ba590f5b642068290"}
Sep 11 12:14:37.885367 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:37.885266    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-986cf2-57cd48794b-t8qcx"
Sep 11 12:14:37.914274 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:37.914202    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-986cf2-65674c9d59-q6f47" podStartSLOduration=1.914177977 podStartE2EDuration="1.914177977s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:14:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:14:37.902990618 +0000 UTC m=+1581.800432361" watchObservedRunningTime="2026-09-11 12:14:37.914177977 +0000 UTC m=+1581.811619726"
Sep 11 12:14:37.934608 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:37.934550    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-986cf2-57cd48794b-t8qcx" podStartSLOduration=1.934532934 podStartE2EDuration="1.934532934s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:14:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:14:37.932796533 +0000 UTC m=+1581.830238289" watchObservedRunningTime="2026-09-11 12:14:37.934532934 +0000 UTC m=+1581.831974679"
Sep 11 12:14:38.577170 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:38.577124    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="650ba6c3-36f7-4f49-9cf7-230d2dcc7d77" path="/var/lib/kubelet/pods/650ba6c3-36f7-4f49-9cf7-230d2dcc7d77/volumes"
Sep 11 12:14:41.434152 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:41.434119    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:41.665492 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:41.665456    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:14:42.252316 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:42.252273    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:42.281384 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:42.281344    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-f7851b-576c644c89-nbcgn"]
Sep 11 12:14:42.282673 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:42.282633    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f7851b-576c644c89-nbcgn" podUID="b5722f04-1a5f-4781-b946-5972261d9a9c" containerName="payload-processing" containerID="cri-o://9a048ed8f208402a25f7c2ab71c02da167da2fda24306f0d0502feba8b4722a9" gracePeriod=30
Sep 11 12:14:42.350806 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:42.350774    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:42.451212 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:42.451180    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:42.459341 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:42.459300    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f7851b-5qmzv"]
Sep 11 12:14:42.463168 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:42.463135    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f7851b-5qmzv"]
Sep 11 12:14:42.550329 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:42.550301    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f7851b-576c644c89-nbcgn"
Sep 11 12:14:42.550934 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:42.550909    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:42.587020 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:42.586709    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9bf573e8-f5df-4d9f-9015-d046efbe6c8d" path="/var/lib/kubelet/pods/9bf573e8-f5df-4d9f-9015-d046efbe6c8d/volumes"
Sep 11 12:14:42.685063 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:42.685026    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5722f04-1a5f-4781-b946-5972261d9a9c-plugins-config-volume\") pod \"b5722f04-1a5f-4781-b946-5972261d9a9c\" (UID: \"b5722f04-1a5f-4781-b946-5972261d9a9c\") "
Sep 11 12:14:42.685235 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:42.685163    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7844s\" (UniqueName: \"kubernetes.io/projected/b5722f04-1a5f-4781-b946-5972261d9a9c-kube-api-access-7844s\") pod \"b5722f04-1a5f-4781-b946-5972261d9a9c\" (UID: \"b5722f04-1a5f-4781-b946-5972261d9a9c\") "
Sep 11 12:14:42.685508 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:42.685474    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5722f04-1a5f-4781-b946-5972261d9a9c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b5722f04-1a5f-4781-b946-5972261d9a9c" (UID: "b5722f04-1a5f-4781-b946-5972261d9a9c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:14:42.687265 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:42.687242    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5722f04-1a5f-4781-b946-5972261d9a9c-kube-api-access-7844s" (OuterVolumeSpecName: "kube-api-access-7844s") pod "b5722f04-1a5f-4781-b946-5972261d9a9c" (UID: "b5722f04-1a5f-4781-b946-5972261d9a9c"). InnerVolumeSpecName "kube-api-access-7844s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:14:42.786437 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:42.786336    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5722f04-1a5f-4781-b946-5972261d9a9c-plugins-config-volume\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:14:42.786437 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:42.786370    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7844s\" (UniqueName: \"kubernetes.io/projected/b5722f04-1a5f-4781-b946-5972261d9a9c-kube-api-access-7844s\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:14:42.817663 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:42.817626    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:42.916113 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:42.916080    2579 generic.go:356] "Generic (PLEG): container finished" podID="b5722f04-1a5f-4781-b946-5972261d9a9c" containerID="9a048ed8f208402a25f7c2ab71c02da167da2fda24306f0d0502feba8b4722a9" exitCode=0
Sep 11 12:14:42.916315 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:42.916132    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f7851b-576c644c89-nbcgn"
Sep 11 12:14:42.916315 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:42.916163    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f7851b-576c644c89-nbcgn" event={"ID":"b5722f04-1a5f-4781-b946-5972261d9a9c","Type":"ContainerDied","Data":"9a048ed8f208402a25f7c2ab71c02da167da2fda24306f0d0502feba8b4722a9"}
Sep 11 12:14:42.916315 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:42.916225    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f7851b-576c644c89-nbcgn" event={"ID":"b5722f04-1a5f-4781-b946-5972261d9a9c","Type":"ContainerDied","Data":"af08e6e6e05712fc8ca5cf1f5074e6cd326e094bf77bbefa6ed28540fe1f8047"}
Sep 11 12:14:42.916315 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:42.916256    2579 scope.go:117] "RemoveContainer" containerID="9a048ed8f208402a25f7c2ab71c02da167da2fda24306f0d0502feba8b4722a9"
Sep 11 12:14:42.926111 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:42.926089    2579 scope.go:117] "RemoveContainer" containerID="9a048ed8f208402a25f7c2ab71c02da167da2fda24306f0d0502feba8b4722a9"
Sep 11 12:14:42.926432 ip-10-0-142-145 kubenswrapper[2579]: E0911 12:14:42.926372    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a048ed8f208402a25f7c2ab71c02da167da2fda24306f0d0502feba8b4722a9\": container with ID starting with 9a048ed8f208402a25f7c2ab71c02da167da2fda24306f0d0502feba8b4722a9 not found: ID does not exist" containerID="9a048ed8f208402a25f7c2ab71c02da167da2fda24306f0d0502feba8b4722a9"
Sep 11 12:14:42.926529 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:42.926427    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a048ed8f208402a25f7c2ab71c02da167da2fda24306f0d0502feba8b4722a9"} err="failed to get container status \"9a048ed8f208402a25f7c2ab71c02da167da2fda24306f0d0502feba8b4722a9\": rpc error: code = NotFound desc = could not find container \"9a048ed8f208402a25f7c2ab71c02da167da2fda24306f0d0502feba8b4722a9\": container with ID starting with 9a048ed8f208402a25f7c2ab71c02da167da2fda24306f0d0502feba8b4722a9 not found: ID does not exist"
Sep 11 12:14:42.942662 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:42.942635    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-f7851b-576c644c89-nbcgn"]
Sep 11 12:14:42.944589 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:42.944572    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-f7851b-576c644c89-nbcgn"]
Sep 11 12:14:44.577003 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:44.576973    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5722f04-1a5f-4781-b946-5972261d9a9c" path="/var/lib/kubelet/pods/b5722f04-1a5f-4781-b946-5972261d9a9c/volumes"
Sep 11 12:14:48.892455 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:48.892420    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-986cf2-65674c9d59-q6f47"
Sep 11 12:14:48.892829 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:48.892496    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-986cf2-57cd48794b-t8qcx"
Sep 11 12:14:49.291539 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:49.291451    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:49.403870 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:49.403836    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:49.693640 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:49.693608    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:54.981785 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:54.981752    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:14:57.151757 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:57.151716    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:59.889486 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:14:59.889452    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:15:00.195588 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:15:00.195501    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-986cf2-29818815-tgqdt"]
Sep 11 12:15:00.195970 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:15:00.195957    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5722f04-1a5f-4781-b946-5972261d9a9c" containerName="payload-processing"
Sep 11 12:15:00.196018 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:15:00.195973    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5722f04-1a5f-4781-b946-5972261d9a9c" containerName="payload-processing"
Sep 11 12:15:00.196058 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:15:00.196053    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5722f04-1a5f-4781-b946-5972261d9a9c" containerName="payload-processing"
Sep 11 12:15:00.199243 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:15:00.199226    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-d844cc-29818815-29fh8"]
Sep 11 12:15:00.199368 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:15:00.199349    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-986cf2-29818815-tgqdt"
Sep 11 12:15:00.202723 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:15:00.202689    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9xkpf\""
Sep 11 12:15:00.202876 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:15:00.202742    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-d844cc-29818815-29fh8"
Sep 11 12:15:00.205342 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:15:00.205315    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-bf9d5c-29818815-qtspn"]
Sep 11 12:15:00.208718 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:15:00.208698    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-bf9d5c-29818815-qtspn"
Sep 11 12:15:00.210844 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:15:00.210821    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-986cf2-29818815-tgqdt"
Sep 11 12:15:00.213483 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:15:00.213460    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-d844cc-29818815-29fh8"]
Sep 11 12:15:00.220309 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:15:00.220281    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-986cf2-29818815-tgqdt"]
Sep 11 12:15:00.224439 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:15:00.224421    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-d844cc-29818815-29fh8"
Sep 11 12:15:00.230068 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:15:00.230023    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-bf9d5c-29818815-qtspn"]
Sep 11 12:15:00.272935 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:15:00.272909    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-bf9d5c-29818815-qtspn"
Sep 11 12:15:00.511199 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:15:00.511167    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-986cf2-29818815-tgqdt"]
Sep 11 12:15:00.513384 ip-10-0-142-145 kubenswrapper[2579]: W0911 12:15:00.513358    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c0da0b7_c20a_450d_8811_3030a7197f07.slice/crio-db504c214fb203bf286648ca6f4fb54a49c70f9b80c74189a6bfb461dd716153 WatchSource:0}: Error finding container db504c214fb203bf286648ca6f4fb54a49c70f9b80c74189a6bfb461dd716153: Status 404 returned error can't find the container with id db504c214fb203bf286648ca6f4fb54a49c70f9b80c74189a6bfb461dd716153
Sep 11 12:15:00.530963 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:15:00.530939    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-d844cc-29818815-29fh8"]
Sep 11 12:15:00.532068 ip-10-0-142-145 kubenswrapper[2579]: W0911 12:15:00.532041    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bf9290f_b75a_46ea_9d49_c838966cda75.slice/crio-9fbcc27ccaf02d017a705424cb4481713e3f6c7c40757f6de7df983d267c79f1 WatchSource:0}: Error finding container 9fbcc27ccaf02d017a705424cb4481713e3f6c7c40757f6de7df983d267c79f1: Status 404 returned error can't find the container with id 9fbcc27ccaf02d017a705424cb4481713e3f6c7c40757f6de7df983d267c79f1
Sep 11 12:15:00.576409 ip-10-0-142-145 kubenswrapper[2579]: W0911 12:15:00.576368    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7286f2e9_a26b_4e00_b0e0_527b240aefae.slice/crio-9fdd461a358ee05ddbdc234ea5f1d5dc164bed9d32bd9e3ecb32a0010edbde1a WatchSource:0}: Error finding container 9fdd461a358ee05ddbdc234ea5f1d5dc164bed9d32bd9e3ecb32a0010edbde1a: Status 404 returned error can't find the container with id 9fdd461a358ee05ddbdc234ea5f1d5dc164bed9d32bd9e3ecb32a0010edbde1a
Sep 11 12:15:00.579055 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:15:00.579035    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-bf9d5c-29818815-qtspn"]
Sep 11 12:15:01.020779 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:15:01.020737    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-bf9d5c-29818815-qtspn" event={"ID":"7286f2e9-a26b-4e00-b0e0-527b240aefae","Type":"ContainerStarted","Data":"28174919688c0dca8e6be4c9e8d293768a6f685f533d3d388cc496ec00dbd857"}
Sep 11 12:15:01.021223 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:15:01.020846    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-bf9d5c-29818815-qtspn" event={"ID":"7286f2e9-a26b-4e00-b0e0-527b240aefae","Type":"ContainerStarted","Data":"9fdd461a358ee05ddbdc234ea5f1d5dc164bed9d32bd9e3ecb32a0010edbde1a"}
Sep 11 12:15:01.022582 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:15:01.022553    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-d844cc-29818815-29fh8" event={"ID":"5bf9290f-b75a-46ea-9d49-c838966cda75","Type":"ContainerStarted","Data":"4ece65f372430b9bc7fdb0fc6d74752349d58c587b2d1d4a446b07de7f9b9bba"}
Sep 11 12:15:01.022723 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:15:01.022608    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-d844cc-29818815-29fh8" event={"ID":"5bf9290f-b75a-46ea-9d49-c838966cda75","Type":"ContainerStarted","Data":"9fbcc27ccaf02d017a705424cb4481713e3f6c7c40757f6de7df983d267c79f1"}
Sep 11 12:15:01.024301 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:15:01.024272    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-986cf2-29818815-tgqdt" event={"ID":"3c0da0b7-c20a-450d-8811-3030a7197f07","Type":"ContainerStarted","Data":"97398cc2d2652c775caebebaa438d1b97372266292dff20de781a08c01ee0300"}
Sep 11 12:15:01.024420 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:15:01.024322    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-986cf2-29818815-tgqdt" event={"ID":"3c0da0b7-c20a-450d-8811-3030a7197f07","Type":"ContainerStarted","Data":"db504c214fb203bf286648ca6f4fb54a49c70f9b80c74189a6bfb461dd716153"}
Sep 11 12:15:01.038655 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:15:01.038573    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-bf9d5c-29818815-qtspn" podStartSLOduration=1.038558613 podStartE2EDuration="1.038558613s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:15:01.03751037 +0000 UTC m=+1604.934952116" watchObservedRunningTime="2026-09-11 12:15:01.038558613 +0000 UTC m=+1604.936000352"
Sep 11 12:15:01.068878 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:15:01.068821    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-986cf2-29818815-tgqdt" podStartSLOduration=1.068802109 podStartE2EDuration="1.068802109s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:15:01.053748287 +0000 UTC m=+1604.951190032" watchObservedRunningTime="2026-09-11 12:15:01.068802109 +0000 UTC m=+1604.966243855"
Sep 11 12:15:01.069746 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:15:01.069711    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-d844cc-29818815-29fh8" podStartSLOduration=1.069700386 podStartE2EDuration="1.069700386s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:15:01.06841291 +0000 UTC m=+1604.965854654" watchObservedRunningTime="2026-09-11 12:15:01.069700386 +0000 UTC m=+1604.967142132"
Sep 11 12:15:02.031862 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:15:02.031766    2579 generic.go:356] "Generic (PLEG): container finished" podID="5bf9290f-b75a-46ea-9d49-c838966cda75" containerID="4ece65f372430b9bc7fdb0fc6d74752349d58c587b2d1d4a446b07de7f9b9bba" exitCode=0
Sep 11 12:15:02.032290 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:15:02.031862    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-d844cc-29818815-29fh8" event={"ID":"5bf9290f-b75a-46ea-9d49-c838966cda75","Type":"ContainerDied","Data":"4ece65f372430b9bc7fdb0fc6d74752349d58c587b2d1d4a446b07de7f9b9bba"}
Sep 11 12:15:02.033526 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:15:02.033500    2579 generic.go:356] "Generic (PLEG): container finished" podID="3c0da0b7-c20a-450d-8811-3030a7197f07" containerID="97398cc2d2652c775caebebaa438d1b97372266292dff20de781a08c01ee0300" exitCode=0
Sep 11 12:15:02.033649 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:15:02.033581    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-986cf2-29818815-tgqdt" event={"ID":"3c0da0b7-c20a-450d-8811-3030a7197f07","Type":"ContainerDied","Data":"97398cc2d2652c775caebebaa438d1b97372266292dff20de781a08c01ee0300"}
Sep 11 12:15:02.035075 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:15:02.035051    2579 generic.go:356] "Generic (PLEG): container finished" podID="7286f2e9-a26b-4e00-b0e0-527b240aefae" containerID="28174919688c0dca8e6be4c9e8d293768a6f685f533d3d388cc496ec00dbd857" exitCode=0
Sep 11 12:15:02.035180 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:15:02.035128    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-bf9d5c-29818815-qtspn" event={"ID":"7286f2e9-a26b-4e00-b0e0-527b240aefae","Type":"ContainerDied","Data":"28174919688c0dca8e6be4c9e8d293768a6f685f533d3d388cc496ec00dbd857"}
Sep 11 12:15:03.239751 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:15:03.239717    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-bf9d5c-29818815-qtspn"
Sep 11 12:15:03.244208 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:15:03.244186    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-986cf2-29818815-tgqdt"
Sep 11 12:15:03.247775 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:15:03.247752    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-d844cc-29818815-29fh8"
Sep 11 12:15:04.049028 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:15:04.049003    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-d844cc-29818815-29fh8"
Sep 11 12:15:04.049028 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:15:04.049011    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-d844cc-29818815-29fh8" event={"ID":"5bf9290f-b75a-46ea-9d49-c838966cda75","Type":"ContainerDied","Data":"9fbcc27ccaf02d017a705424cb4481713e3f6c7c40757f6de7df983d267c79f1"}
Sep 11 12:15:04.049288 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:15:04.049056    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9fbcc27ccaf02d017a705424cb4481713e3f6c7c40757f6de7df983d267c79f1"
Sep 11 12:15:04.050552 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:15:04.050529    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-986cf2-29818815-tgqdt"
Sep 11 12:15:04.050682 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:15:04.050567    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-986cf2-29818815-tgqdt" event={"ID":"3c0da0b7-c20a-450d-8811-3030a7197f07","Type":"ContainerDied","Data":"db504c214fb203bf286648ca6f4fb54a49c70f9b80c74189a6bfb461dd716153"}
Sep 11 12:15:04.050682 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:15:04.050614    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="db504c214fb203bf286648ca6f4fb54a49c70f9b80c74189a6bfb461dd716153"
Sep 11 12:15:04.051955 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:15:04.051921    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-bf9d5c-29818815-qtspn" event={"ID":"7286f2e9-a26b-4e00-b0e0-527b240aefae","Type":"ContainerDied","Data":"9fdd461a358ee05ddbdc234ea5f1d5dc164bed9d32bd9e3ecb32a0010edbde1a"}
Sep 11 12:15:04.052070 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:15:04.051971    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9fdd461a358ee05ddbdc234ea5f1d5dc164bed9d32bd9e3ecb32a0010edbde1a"
Sep 11 12:15:04.052070 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:15:04.051943    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-bf9d5c-29818815-qtspn"
Sep 11 12:15:05.038359 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:15:05.038331    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-986cf2-65674c9d59-q6f47_4e15f705-fc9a-4af7-b05a-5c359c81ac87/payload-processing/0.log"
Sep 11 12:15:18.575045 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:15:18.574997    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:15:22.466780 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:15:22.466745    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:15:47.266588 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:15:47.266544    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:15:49.499893 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:15:49.499855    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:15:49.669242 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:15:49.669206    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:15:56.180318 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:15:56.180291    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-986cf2-65674c9d59-q6f47_4e15f705-fc9a-4af7-b05a-5c359c81ac87/payload-processing/0.log"
Sep 11 12:15:56.447879 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:15:56.447781    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:15:56.764614 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:15:56.764527    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:15:56.861404 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:15:56.861367    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:15:59.440414 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:15:59.440372    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:15:59.958589 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:15:59.958535    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:16:00.579635 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:00.579601    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5c1bd9-747f555679-p2bmk"]
Sep 11 12:16:00.580120 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:00.580052    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5bf9290f-b75a-46ea-9d49-c838966cda75" containerName="cleanup"
Sep 11 12:16:00.580120 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:00.580065    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="5bf9290f-b75a-46ea-9d49-c838966cda75" containerName="cleanup"
Sep 11 12:16:00.580120 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:00.580080    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c0da0b7-c20a-450d-8811-3030a7197f07" containerName="cleanup"
Sep 11 12:16:00.580120 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:00.580087    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c0da0b7-c20a-450d-8811-3030a7197f07" containerName="cleanup"
Sep 11 12:16:00.580120 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:00.580099    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7286f2e9-a26b-4e00-b0e0-527b240aefae" containerName="cleanup"
Sep 11 12:16:00.580120 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:00.580105    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7286f2e9-a26b-4e00-b0e0-527b240aefae" containerName="cleanup"
Sep 11 12:16:00.580488 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:00.580194    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="7286f2e9-a26b-4e00-b0e0-527b240aefae" containerName="cleanup"
Sep 11 12:16:00.580488 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:00.580203    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c0da0b7-c20a-450d-8811-3030a7197f07" containerName="cleanup"
Sep 11 12:16:00.580488 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:00.580209    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="5bf9290f-b75a-46ea-9d49-c838966cda75" containerName="cleanup"
Sep 11 12:16:00.583828 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:00.583805    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5c1bd9-747f555679-p2bmk"
Sep 11 12:16:00.586809 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:00.586786    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 11 12:16:00.587101 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:00.586796    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-rrxjh\""
Sep 11 12:16:00.587238 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:00.586913    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 12:16:00.587238 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:00.586963    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 11 12:16:00.587407 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:00.587372    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-5c1bd9-serving-cert\""
Sep 11 12:16:00.588956 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:00.588932    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5c1bd9-747f555679-p2bmk"]
Sep 11 12:16:00.616674 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:00.616622    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:16:00.664894 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:00.664859    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlrtv\" (UniqueName: \"kubernetes.io/projected/5428e182-60c8-4507-95dd-fbec8ad87ca8-kube-api-access-hlrtv\") pod \"maas-api-e2e-ipp-cleanup-5c1bd9-747f555679-p2bmk\" (UID: \"5428e182-60c8-4507-95dd-fbec8ad87ca8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5c1bd9-747f555679-p2bmk"
Sep 11 12:16:00.665067 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:00.664900    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5428e182-60c8-4507-95dd-fbec8ad87ca8-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-5c1bd9-747f555679-p2bmk\" (UID: \"5428e182-60c8-4507-95dd-fbec8ad87ca8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5c1bd9-747f555679-p2bmk"
Sep 11 12:16:00.665067 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:00.664926    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5428e182-60c8-4507-95dd-fbec8ad87ca8-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-5c1bd9-747f555679-p2bmk\" (UID: \"5428e182-60c8-4507-95dd-fbec8ad87ca8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5c1bd9-747f555679-p2bmk"
Sep 11 12:16:00.665067 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:00.665034    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5428e182-60c8-4507-95dd-fbec8ad87ca8-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-5c1bd9-747f555679-p2bmk\" (UID: \"5428e182-60c8-4507-95dd-fbec8ad87ca8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5c1bd9-747f555679-p2bmk"
Sep 11 12:16:00.727090 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:00.727029    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:16:00.765658 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:00.765619    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5428e182-60c8-4507-95dd-fbec8ad87ca8-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-5c1bd9-747f555679-p2bmk\" (UID: \"5428e182-60c8-4507-95dd-fbec8ad87ca8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5c1bd9-747f555679-p2bmk"
Sep 11 12:16:00.765820 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:00.765677    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5428e182-60c8-4507-95dd-fbec8ad87ca8-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-5c1bd9-747f555679-p2bmk\" (UID: \"5428e182-60c8-4507-95dd-fbec8ad87ca8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5c1bd9-747f555679-p2bmk"
Sep 11 12:16:00.765820 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:00.765755    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5428e182-60c8-4507-95dd-fbec8ad87ca8-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-5c1bd9-747f555679-p2bmk\" (UID: \"5428e182-60c8-4507-95dd-fbec8ad87ca8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5c1bd9-747f555679-p2bmk"
Sep 11 12:16:00.765944 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:00.765860    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlrtv\" (UniqueName: \"kubernetes.io/projected/5428e182-60c8-4507-95dd-fbec8ad87ca8-kube-api-access-hlrtv\") pod \"maas-api-e2e-ipp-cleanup-5c1bd9-747f555679-p2bmk\" (UID: \"5428e182-60c8-4507-95dd-fbec8ad87ca8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5c1bd9-747f555679-p2bmk"
Sep 11 12:16:00.766377 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:00.766348    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5428e182-60c8-4507-95dd-fbec8ad87ca8-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-5c1bd9-747f555679-p2bmk\" (UID: \"5428e182-60c8-4507-95dd-fbec8ad87ca8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5c1bd9-747f555679-p2bmk"
Sep 11 12:16:00.768302 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:00.768275    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5428e182-60c8-4507-95dd-fbec8ad87ca8-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-5c1bd9-747f555679-p2bmk\" (UID: \"5428e182-60c8-4507-95dd-fbec8ad87ca8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5c1bd9-747f555679-p2bmk"
Sep 11 12:16:00.768517 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:00.768498    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5428e182-60c8-4507-95dd-fbec8ad87ca8-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-5c1bd9-747f555679-p2bmk\" (UID: \"5428e182-60c8-4507-95dd-fbec8ad87ca8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5c1bd9-747f555679-p2bmk"
Sep 11 12:16:00.797027 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:00.796993    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlrtv\" (UniqueName: \"kubernetes.io/projected/5428e182-60c8-4507-95dd-fbec8ad87ca8-kube-api-access-hlrtv\") pod \"maas-api-e2e-ipp-cleanup-5c1bd9-747f555679-p2bmk\" (UID: \"5428e182-60c8-4507-95dd-fbec8ad87ca8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5c1bd9-747f555679-p2bmk"
Sep 11 12:16:00.897611 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:00.897573    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5c1bd9-747f555679-p2bmk"
Sep 11 12:16:01.074295 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:01.074255    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:16:01.182041 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:01.181960    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5c1bd9-747f555679-p2bmk"]
Sep 11 12:16:01.186247 ip-10-0-142-145 kubenswrapper[2579]: W0911 12:16:01.186218    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5428e182_60c8_4507_95dd_fbec8ad87ca8.slice/crio-ad0239c1fc3f96fae7e88cd844242d19fb4e280e9c575cdd5899673894aae028 WatchSource:0}: Error finding container ad0239c1fc3f96fae7e88cd844242d19fb4e280e9c575cdd5899673894aae028: Status 404 returned error can't find the container with id ad0239c1fc3f96fae7e88cd844242d19fb4e280e9c575cdd5899673894aae028
Sep 11 12:16:01.187719 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:01.187699    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 12:16:01.203193 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:01.203163    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:16:01.383845 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:01.383808    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5c1bd9-747f555679-p2bmk" event={"ID":"5428e182-60c8-4507-95dd-fbec8ad87ca8","Type":"ContainerStarted","Data":"ad0239c1fc3f96fae7e88cd844242d19fb4e280e9c575cdd5899673894aae028"}
Sep 11 12:16:01.703571 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:01.703536    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:16:02.391071 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:02.391040    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5c1bd9-747f555679-p2bmk" event={"ID":"5428e182-60c8-4507-95dd-fbec8ad87ca8","Type":"ContainerStarted","Data":"3228aedefcf8ac2f824360ec60d740a032667e6c99f36f980b8c41d7bf929045"}
Sep 11 12:16:02.391286 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:02.391112    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5c1bd9-747f555679-p2bmk"
Sep 11 12:16:02.411191 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:02.411134    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5c1bd9-747f555679-p2bmk" podStartSLOduration=2.065973338 podStartE2EDuration="2.411113885s" totalImagesPullingTime="345.140547ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:16:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:16:02.408721433 +0000 UTC m=+1666.306163183" watchObservedRunningTime="2026-09-11 12:16:02.411113885 +0000 UTC m=+1666.308555633"
Sep 11 12:16:08.403652 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:08.403619    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5c1bd9-747f555679-p2bmk"
Sep 11 12:16:09.813335 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:09.813298    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:16:24.409055 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:24.409011    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:16:31.199153 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:31.199112    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5c1bd9-747f555679-p2bmk"]
Sep 11 12:16:31.199631 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:31.199380    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5c1bd9-747f555679-p2bmk" podUID="5428e182-60c8-4507-95dd-fbec8ad87ca8" containerName="maas-api" containerID="cri-o://3228aedefcf8ac2f824360ec60d740a032667e6c99f36f980b8c41d7bf929045" gracePeriod=30
Sep 11 12:16:31.322360 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:31.322321    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:16:31.415223 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:31.415185    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:16:31.468429 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:31.468283    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5c1bd9-747f555679-p2bmk"
Sep 11 12:16:31.499724 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:31.499651    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:16:31.560811 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:31.560772    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hlrtv\" (UniqueName: \"kubernetes.io/projected/5428e182-60c8-4507-95dd-fbec8ad87ca8-kube-api-access-hlrtv\") pod \"5428e182-60c8-4507-95dd-fbec8ad87ca8\" (UID: \"5428e182-60c8-4507-95dd-fbec8ad87ca8\") "
Sep 11 12:16:31.561021 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:31.560890    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5428e182-60c8-4507-95dd-fbec8ad87ca8-maas-api-tls\") pod \"5428e182-60c8-4507-95dd-fbec8ad87ca8\" (UID: \"5428e182-60c8-4507-95dd-fbec8ad87ca8\") "
Sep 11 12:16:31.561021 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:31.560945    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5428e182-60c8-4507-95dd-fbec8ad87ca8-metrics-certs\") pod \"5428e182-60c8-4507-95dd-fbec8ad87ca8\" (UID: \"5428e182-60c8-4507-95dd-fbec8ad87ca8\") "
Sep 11 12:16:31.561021 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:31.560996    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5428e182-60c8-4507-95dd-fbec8ad87ca8-openshift-service-ca\") pod \"5428e182-60c8-4507-95dd-fbec8ad87ca8\" (UID: \"5428e182-60c8-4507-95dd-fbec8ad87ca8\") "
Sep 11 12:16:31.561444 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:31.561412    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5428e182-60c8-4507-95dd-fbec8ad87ca8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5428e182-60c8-4507-95dd-fbec8ad87ca8" (UID: "5428e182-60c8-4507-95dd-fbec8ad87ca8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:16:31.561685 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:31.561661    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5428e182-60c8-4507-95dd-fbec8ad87ca8-openshift-service-ca\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:16:31.562126 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:31.562099    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:16:31.563203 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:31.563180    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5428e182-60c8-4507-95dd-fbec8ad87ca8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5428e182-60c8-4507-95dd-fbec8ad87ca8" (UID: "5428e182-60c8-4507-95dd-fbec8ad87ca8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:16:31.563285 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:31.563247    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5428e182-60c8-4507-95dd-fbec8ad87ca8-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "5428e182-60c8-4507-95dd-fbec8ad87ca8" (UID: "5428e182-60c8-4507-95dd-fbec8ad87ca8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:16:31.563285 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:31.563259    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5428e182-60c8-4507-95dd-fbec8ad87ca8-kube-api-access-hlrtv" (OuterVolumeSpecName: "kube-api-access-hlrtv") pod "5428e182-60c8-4507-95dd-fbec8ad87ca8" (UID: "5428e182-60c8-4507-95dd-fbec8ad87ca8"). InnerVolumeSpecName "kube-api-access-hlrtv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:16:31.579412 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:31.575387    2579 generic.go:356] "Generic (PLEG): container finished" podID="5428e182-60c8-4507-95dd-fbec8ad87ca8" containerID="3228aedefcf8ac2f824360ec60d740a032667e6c99f36f980b8c41d7bf929045" exitCode=0
Sep 11 12:16:31.579412 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:31.575426    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5c1bd9-747f555679-p2bmk" event={"ID":"5428e182-60c8-4507-95dd-fbec8ad87ca8","Type":"ContainerDied","Data":"3228aedefcf8ac2f824360ec60d740a032667e6c99f36f980b8c41d7bf929045"}
Sep 11 12:16:31.579412 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:31.575454    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5c1bd9-747f555679-p2bmk"
Sep 11 12:16:31.579412 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:31.575496    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5c1bd9-747f555679-p2bmk" event={"ID":"5428e182-60c8-4507-95dd-fbec8ad87ca8","Type":"ContainerDied","Data":"ad0239c1fc3f96fae7e88cd844242d19fb4e280e9c575cdd5899673894aae028"}
Sep 11 12:16:31.579412 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:31.575526    2579 scope.go:117] "RemoveContainer" containerID="3228aedefcf8ac2f824360ec60d740a032667e6c99f36f980b8c41d7bf929045"
Sep 11 12:16:31.589296 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:31.589272    2579 scope.go:117] "RemoveContainer" containerID="3228aedefcf8ac2f824360ec60d740a032667e6c99f36f980b8c41d7bf929045"
Sep 11 12:16:31.589637 ip-10-0-142-145 kubenswrapper[2579]: E0911 12:16:31.589614    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3228aedefcf8ac2f824360ec60d740a032667e6c99f36f980b8c41d7bf929045\": container with ID starting with 3228aedefcf8ac2f824360ec60d740a032667e6c99f36f980b8c41d7bf929045 not found: ID does not exist" containerID="3228aedefcf8ac2f824360ec60d740a032667e6c99f36f980b8c41d7bf929045"
Sep 11 12:16:31.589717 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:31.589644    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3228aedefcf8ac2f824360ec60d740a032667e6c99f36f980b8c41d7bf929045"} err="failed to get container status \"3228aedefcf8ac2f824360ec60d740a032667e6c99f36f980b8c41d7bf929045\": rpc error: code = NotFound desc = could not find container \"3228aedefcf8ac2f824360ec60d740a032667e6c99f36f980b8c41d7bf929045\": container with ID starting with 3228aedefcf8ac2f824360ec60d740a032667e6c99f36f980b8c41d7bf929045 not found: ID does not exist"
Sep 11 12:16:31.609923 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:31.609891    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5c1bd9-747f555679-p2bmk"]
Sep 11 12:16:31.612798 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:31.612774    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5c1bd9-747f555679-p2bmk"]
Sep 11 12:16:31.662723 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:31.662686    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hlrtv\" (UniqueName: \"kubernetes.io/projected/5428e182-60c8-4507-95dd-fbec8ad87ca8-kube-api-access-hlrtv\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:16:31.662723 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:31.662717    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5428e182-60c8-4507-95dd-fbec8ad87ca8-maas-api-tls\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:16:31.662723 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:31.662730    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5428e182-60c8-4507-95dd-fbec8ad87ca8-metrics-certs\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:16:31.850723 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:31.850686    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:16:32.576687 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:32.576647    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5428e182-60c8-4507-95dd-fbec8ad87ca8" path="/var/lib/kubelet/pods/5428e182-60c8-4507-95dd-fbec8ad87ca8/volumes"
Sep 11 12:16:35.662256 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:35.662219    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:16:37.949532 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:37.949493    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:16:38.052807 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:38.052763    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:16:38.199946 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:38.199862    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:16:41.782896 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:41.782808    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-bf9d5c-zrktg"]
Sep 11 12:16:41.783293 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:41.783279    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5428e182-60c8-4507-95dd-fbec8ad87ca8" containerName="maas-api"
Sep 11 12:16:41.783343 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:41.783294    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="5428e182-60c8-4507-95dd-fbec8ad87ca8" containerName="maas-api"
Sep 11 12:16:41.783379 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:41.783369    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="5428e182-60c8-4507-95dd-fbec8ad87ca8" containerName="maas-api"
Sep 11 12:16:41.788014 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:41.787995    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-bf9d5c-zrktg"
Sep 11 12:16:41.791275 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:41.791248    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9xkpf\""
Sep 11 12:16:41.791495 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:41.791370    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 12:16:41.796533 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:41.795994    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-bf9d5c-zrktg"]
Sep 11 12:16:41.871500 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:41.871458    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/15cb5605-2e82-4797-9b32-6f61675e2a45-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-bf9d5c-zrktg\" (UID: \"15cb5605-2e82-4797-9b32-6f61675e2a45\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-bf9d5c-zrktg"
Sep 11 12:16:41.972780 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:41.972746    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/15cb5605-2e82-4797-9b32-6f61675e2a45-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-bf9d5c-zrktg\" (UID: \"15cb5605-2e82-4797-9b32-6f61675e2a45\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-bf9d5c-zrktg"
Sep 11 12:16:41.973355 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:41.973336    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/15cb5605-2e82-4797-9b32-6f61675e2a45-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-bf9d5c-zrktg\" (UID: \"15cb5605-2e82-4797-9b32-6f61675e2a45\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-bf9d5c-zrktg"
Sep 11 12:16:42.035915 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:42.035830    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:16:42.100887 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:42.100848    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-bf9d5c-zrktg"
Sep 11 12:16:42.374114 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:42.374083    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-bf9d5c-zrktg"]
Sep 11 12:16:42.376227 ip-10-0-142-145 kubenswrapper[2579]: W0911 12:16:42.376196    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15cb5605_2e82_4797_9b32_6f61675e2a45.slice/crio-fd5277a84516d9efd1ea846e2cdb2c2f1785e04150761248d615865618de04db WatchSource:0}: Error finding container fd5277a84516d9efd1ea846e2cdb2c2f1785e04150761248d615865618de04db: Status 404 returned error can't find the container with id fd5277a84516d9efd1ea846e2cdb2c2f1785e04150761248d615865618de04db
Sep 11 12:16:42.639256 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:42.639166    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-bf9d5c-zrktg" event={"ID":"15cb5605-2e82-4797-9b32-6f61675e2a45","Type":"ContainerStarted","Data":"fd5277a84516d9efd1ea846e2cdb2c2f1785e04150761248d615865618de04db"}
Sep 11 12:16:43.647130 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:43.647092    2579 generic.go:356] "Generic (PLEG): container finished" podID="15cb5605-2e82-4797-9b32-6f61675e2a45" containerID="60a0a5aa8e567a29e2f654acb94d1f6a1ee7a149bc5bda8c6d8d0f52c294d1c4" exitCode=0
Sep 11 12:16:43.647560 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:43.647172    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-bf9d5c-zrktg" event={"ID":"15cb5605-2e82-4797-9b32-6f61675e2a45","Type":"ContainerDied","Data":"60a0a5aa8e567a29e2f654acb94d1f6a1ee7a149bc5bda8c6d8d0f52c294d1c4"}
Sep 11 12:16:44.797732 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:44.797709    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-bf9d5c-zrktg"
Sep 11 12:16:44.903790 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:44.903682    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/15cb5605-2e82-4797-9b32-6f61675e2a45-maas-api-service-ca\") pod \"15cb5605-2e82-4797-9b32-6f61675e2a45\" (UID: \"15cb5605-2e82-4797-9b32-6f61675e2a45\") "
Sep 11 12:16:44.904047 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:44.904023    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15cb5605-2e82-4797-9b32-6f61675e2a45-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "15cb5605-2e82-4797-9b32-6f61675e2a45" (UID: "15cb5605-2e82-4797-9b32-6f61675e2a45"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:16:44.904205 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:44.904193    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/15cb5605-2e82-4797-9b32-6f61675e2a45-maas-api-service-ca\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:16:45.660093 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:45.660062    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-bf9d5c-zrktg"
Sep 11 12:16:45.660287 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:45.660108    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-bf9d5c-zrktg" event={"ID":"15cb5605-2e82-4797-9b32-6f61675e2a45","Type":"ContainerDied","Data":"fd5277a84516d9efd1ea846e2cdb2c2f1785e04150761248d615865618de04db"}
Sep 11 12:16:45.660287 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:45.660154    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fd5277a84516d9efd1ea846e2cdb2c2f1785e04150761248d615865618de04db"
Sep 11 12:16:55.077369 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:55.077329    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:16:57.388275 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:57.388238    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-bf9d5c-29818815-qtspn"]
Sep 11 12:16:57.391285 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:57.391256    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:16:57.391714 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:57.391693    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-bf9d5c-29818815-qtspn"]
Sep 11 12:16:57.422815 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:57.422784    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-bf9d5c-5bc4c96fdb-fjvpp"]
Sep 11 12:16:57.423068 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:57.423042    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-bf9d5c-5bc4c96fdb-fjvpp" podUID="4f18bfb8-3568-4039-8a1f-c4017c6beab7" containerName="payload-pre-processing" containerID="cri-o://2b02468ea580e1023de64d528ef0c937f3eaf5e424221ea34f113e43521eafb5" gracePeriod=30
Sep 11 12:16:57.462327 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:57.462256    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:16:57.565813 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:57.565774    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:16:57.605224 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:57.605171    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-bf9d5c-zrktg"]
Sep 11 12:16:57.609532 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:57.609506    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-bf9d5c-zrktg"]
Sep 11 12:16:57.632104 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:57.632071    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:16:57.680142 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:57.680111    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-bf9d5c-5bc4c96fdb-fjvpp"
Sep 11 12:16:57.715700 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:57.715665    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-84bnj\" (UniqueName: \"kubernetes.io/projected/4f18bfb8-3568-4039-8a1f-c4017c6beab7-kube-api-access-84bnj\") pod \"4f18bfb8-3568-4039-8a1f-c4017c6beab7\" (UID: \"4f18bfb8-3568-4039-8a1f-c4017c6beab7\") "
Sep 11 12:16:57.715867 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:57.715716    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f18bfb8-3568-4039-8a1f-c4017c6beab7-plugins-config-volume\") pod \"4f18bfb8-3568-4039-8a1f-c4017c6beab7\" (UID: \"4f18bfb8-3568-4039-8a1f-c4017c6beab7\") "
Sep 11 12:16:57.716127 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:57.716090    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f18bfb8-3568-4039-8a1f-c4017c6beab7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4f18bfb8-3568-4039-8a1f-c4017c6beab7" (UID: "4f18bfb8-3568-4039-8a1f-c4017c6beab7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:16:57.718555 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:57.718522    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f18bfb8-3568-4039-8a1f-c4017c6beab7-kube-api-access-84bnj" (OuterVolumeSpecName: "kube-api-access-84bnj") pod "4f18bfb8-3568-4039-8a1f-c4017c6beab7" (UID: "4f18bfb8-3568-4039-8a1f-c4017c6beab7"). InnerVolumeSpecName "kube-api-access-84bnj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:16:57.730817 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:57.730790    2579 generic.go:356] "Generic (PLEG): container finished" podID="4f18bfb8-3568-4039-8a1f-c4017c6beab7" containerID="2b02468ea580e1023de64d528ef0c937f3eaf5e424221ea34f113e43521eafb5" exitCode=0
Sep 11 12:16:57.730947 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:57.730829    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-bf9d5c-5bc4c96fdb-fjvpp" event={"ID":"4f18bfb8-3568-4039-8a1f-c4017c6beab7","Type":"ContainerDied","Data":"2b02468ea580e1023de64d528ef0c937f3eaf5e424221ea34f113e43521eafb5"}
Sep 11 12:16:57.730947 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:57.730840    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-bf9d5c-5bc4c96fdb-fjvpp"
Sep 11 12:16:57.730947 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:57.730887    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-bf9d5c-5bc4c96fdb-fjvpp" event={"ID":"4f18bfb8-3568-4039-8a1f-c4017c6beab7","Type":"ContainerDied","Data":"33e55b575f0da5ccceb6fa1b29e31fd01322c96373ea03acc1b9597b00972d48"}
Sep 11 12:16:57.730947 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:57.730918    2579 scope.go:117] "RemoveContainer" containerID="2b02468ea580e1023de64d528ef0c937f3eaf5e424221ea34f113e43521eafb5"
Sep 11 12:16:57.742186 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:57.742170    2579 scope.go:117] "RemoveContainer" containerID="2b02468ea580e1023de64d528ef0c937f3eaf5e424221ea34f113e43521eafb5"
Sep 11 12:16:57.742516 ip-10-0-142-145 kubenswrapper[2579]: E0911 12:16:57.742491    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b02468ea580e1023de64d528ef0c937f3eaf5e424221ea34f113e43521eafb5\": container with ID starting with 2b02468ea580e1023de64d528ef0c937f3eaf5e424221ea34f113e43521eafb5 not found: ID does not exist" containerID="2b02468ea580e1023de64d528ef0c937f3eaf5e424221ea34f113e43521eafb5"
Sep 11 12:16:57.742577 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:57.742525    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b02468ea580e1023de64d528ef0c937f3eaf5e424221ea34f113e43521eafb5"} err="failed to get container status \"2b02468ea580e1023de64d528ef0c937f3eaf5e424221ea34f113e43521eafb5\": rpc error: code = NotFound desc = could not find container \"2b02468ea580e1023de64d528ef0c937f3eaf5e424221ea34f113e43521eafb5\": container with ID starting with 2b02468ea580e1023de64d528ef0c937f3eaf5e424221ea34f113e43521eafb5 not found: ID does not exist"
Sep 11 12:16:57.766707 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:57.766663    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-bf9d5c-5bc4c96fdb-fjvpp"]
Sep 11 12:16:57.774019 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:57.773993    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-bf9d5c-5bc4c96fdb-fjvpp"]
Sep 11 12:16:57.817293 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:57.817257    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-84bnj\" (UniqueName: \"kubernetes.io/projected/4f18bfb8-3568-4039-8a1f-c4017c6beab7-kube-api-access-84bnj\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:16:57.817293 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:57.817283    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f18bfb8-3568-4039-8a1f-c4017c6beab7-plugins-config-volume\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:16:58.029925 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:58.029836    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:16:58.576692 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:58.576653    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15cb5605-2e82-4797-9b32-6f61675e2a45" path="/var/lib/kubelet/pods/15cb5605-2e82-4797-9b32-6f61675e2a45/volumes"
Sep 11 12:16:58.577228 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:58.577032    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f18bfb8-3568-4039-8a1f-c4017c6beab7" path="/var/lib/kubelet/pods/4f18bfb8-3568-4039-8a1f-c4017c6beab7/volumes"
Sep 11 12:16:58.577291 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:16:58.577281    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7286f2e9-a26b-4e00-b0e0-527b240aefae" path="/var/lib/kubelet/pods/7286f2e9-a26b-4e00-b0e0-527b240aefae/volumes"
Sep 11 12:17:03.081855 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:17:03.081817    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:17:08.362588 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:17:08.362548    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:17:16.790085 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:17:16.790048    2579 scope.go:117] "RemoveContainer" containerID="30ce195b4b161b45c69ffa3610ba8549609f6ee9b48e93beed7379455fd0b115"
Sep 11 12:17:40.080132 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:17:40.080093    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:17:40.245941 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:17:40.245880    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:17:44.744609 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:17:44.744575    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:17:46.724050 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:17:46.724018    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:18:01.241074 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:01.241023    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:18:01.343269 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:01.343231    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:18:01.490172 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:01.490136    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4"]
Sep 11 12:18:01.490499 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:01.490423    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4" podUID="d361df08-ce57-4396-ab92-a869643ddb40" containerName="istio-proxy" containerID="cri-o://c764a942ba0752066d3e2d5abdcbd085785f63a7bb4305412d18c2f0d5366d96" gracePeriod=30
Sep 11 12:18:01.538043 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:01.537947    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:18:01.873409 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:01.873367    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-986cf2-h8pzf"]
Sep 11 12:18:01.873915 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:01.873900    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f18bfb8-3568-4039-8a1f-c4017c6beab7" containerName="payload-pre-processing"
Sep 11 12:18:01.873966 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:01.873917    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f18bfb8-3568-4039-8a1f-c4017c6beab7" containerName="payload-pre-processing"
Sep 11 12:18:01.873966 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:01.873930    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15cb5605-2e82-4797-9b32-6f61675e2a45" containerName="revoke-keys"
Sep 11 12:18:01.873966 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:01.873936    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="15cb5605-2e82-4797-9b32-6f61675e2a45" containerName="revoke-keys"
Sep 11 12:18:01.874137 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:01.874125    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="15cb5605-2e82-4797-9b32-6f61675e2a45" containerName="revoke-keys"
Sep 11 12:18:01.874188 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:01.874142    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f18bfb8-3568-4039-8a1f-c4017c6beab7" containerName="payload-pre-processing"
Sep 11 12:18:01.877470 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:01.877451    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-986cf2-h8pzf"
Sep 11 12:18:01.880280 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:01.880255    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 12:18:01.880280 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:01.880271    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9xkpf\""
Sep 11 12:18:01.884558 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:01.884529    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-986cf2-h8pzf"]
Sep 11 12:18:01.964227 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:01.964189    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3205c4be-9d94-4e23-84c1-99c56e0e38cb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-986cf2-h8pzf\" (UID: \"3205c4be-9d94-4e23-84c1-99c56e0e38cb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-986cf2-h8pzf"
Sep 11 12:18:01.973285 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:01.973254    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:18:02.065154 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:02.065116    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3205c4be-9d94-4e23-84c1-99c56e0e38cb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-986cf2-h8pzf\" (UID: \"3205c4be-9d94-4e23-84c1-99c56e0e38cb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-986cf2-h8pzf"
Sep 11 12:18:02.065782 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:02.065762    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3205c4be-9d94-4e23-84c1-99c56e0e38cb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-986cf2-h8pzf\" (UID: \"3205c4be-9d94-4e23-84c1-99c56e0e38cb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-986cf2-h8pzf"
Sep 11 12:18:02.190753 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:02.190657    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-986cf2-h8pzf"
Sep 11 12:18:02.462801 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:02.462771    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-986cf2-h8pzf"]
Sep 11 12:18:02.464595 ip-10-0-142-145 kubenswrapper[2579]: W0911 12:18:02.464569    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3205c4be_9d94_4e23_84c1_99c56e0e38cb.slice/crio-c22edf26cb4098bf895bfa7653e202a766d1945564b3c66fa96372c132afbddb WatchSource:0}: Error finding container c22edf26cb4098bf895bfa7653e202a766d1945564b3c66fa96372c132afbddb: Status 404 returned error can't find the container with id c22edf26cb4098bf895bfa7653e202a766d1945564b3c66fa96372c132afbddb
Sep 11 12:18:03.117775 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:03.117736    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-986cf2-h8pzf" event={"ID":"3205c4be-9d94-4e23-84c1-99c56e0e38cb","Type":"ContainerStarted","Data":"f9ec99a04f2b6e3fe5099b0fd3a2f5780aa36cf0a7c4a0ca9ae731e0839bab3a"}
Sep 11 12:18:03.117972 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:03.117800    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-986cf2-h8pzf" event={"ID":"3205c4be-9d94-4e23-84c1-99c56e0e38cb","Type":"ContainerStarted","Data":"c22edf26cb4098bf895bfa7653e202a766d1945564b3c66fa96372c132afbddb"}
Sep 11 12:18:03.134581 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:03.134522    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-986cf2-h8pzf" podStartSLOduration=2.134502933 podStartE2EDuration="2.134502933s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:18:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:18:03.133290903 +0000 UTC m=+1787.030732651" watchObservedRunningTime="2026-09-11 12:18:03.134502933 +0000 UTC m=+1787.031944678"
Sep 11 12:18:04.125341 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:04.125303    2579 generic.go:356] "Generic (PLEG): container finished" podID="3205c4be-9d94-4e23-84c1-99c56e0e38cb" containerID="f9ec99a04f2b6e3fe5099b0fd3a2f5780aa36cf0a7c4a0ca9ae731e0839bab3a" exitCode=0
Sep 11 12:18:04.125787 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:04.125408    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-986cf2-h8pzf" event={"ID":"3205c4be-9d94-4e23-84c1-99c56e0e38cb","Type":"ContainerDied","Data":"f9ec99a04f2b6e3fe5099b0fd3a2f5780aa36cf0a7c4a0ca9ae731e0839bab3a"}
Sep 11 12:18:05.264504 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:05.264481    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-986cf2-h8pzf"
Sep 11 12:18:05.298305 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:05.298272    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3205c4be-9d94-4e23-84c1-99c56e0e38cb-maas-api-service-ca\") pod \"3205c4be-9d94-4e23-84c1-99c56e0e38cb\" (UID: \"3205c4be-9d94-4e23-84c1-99c56e0e38cb\") "
Sep 11 12:18:05.298637 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:05.298613    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3205c4be-9d94-4e23-84c1-99c56e0e38cb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3205c4be-9d94-4e23-84c1-99c56e0e38cb" (UID: "3205c4be-9d94-4e23-84c1-99c56e0e38cb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:18:05.298744 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:05.298732    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3205c4be-9d94-4e23-84c1-99c56e0e38cb-maas-api-service-ca\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:18:06.138600 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:06.138566    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-986cf2-h8pzf"
Sep 11 12:18:06.138805 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:06.138607    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-986cf2-h8pzf" event={"ID":"3205c4be-9d94-4e23-84c1-99c56e0e38cb","Type":"ContainerDied","Data":"c22edf26cb4098bf895bfa7653e202a766d1945564b3c66fa96372c132afbddb"}
Sep 11 12:18:06.138805 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:06.138674    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c22edf26cb4098bf895bfa7653e202a766d1945564b3c66fa96372c132afbddb"
Sep 11 12:18:06.747853 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:06.747818    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4"
Sep 11 12:18:06.816690 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:06.816603    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d361df08-ce57-4396-ab92-a869643ddb40-workload-certs\") pod \"d361df08-ce57-4396-ab92-a869643ddb40\" (UID: \"d361df08-ce57-4396-ab92-a869643ddb40\") "
Sep 11 12:18:06.816690 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:06.816649    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d361df08-ce57-4396-ab92-a869643ddb40-istio-data\") pod \"d361df08-ce57-4396-ab92-a869643ddb40\" (UID: \"d361df08-ce57-4396-ab92-a869643ddb40\") "
Sep 11 12:18:06.816690 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:06.816687    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d361df08-ce57-4396-ab92-a869643ddb40-istio-podinfo\") pod \"d361df08-ce57-4396-ab92-a869643ddb40\" (UID: \"d361df08-ce57-4396-ab92-a869643ddb40\") "
Sep 11 12:18:06.816954 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:06.816720    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d361df08-ce57-4396-ab92-a869643ddb40-credential-socket\") pod \"d361df08-ce57-4396-ab92-a869643ddb40\" (UID: \"d361df08-ce57-4396-ab92-a869643ddb40\") "
Sep 11 12:18:06.816954 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:06.816750    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4ggq8\" (UniqueName: \"kubernetes.io/projected/d361df08-ce57-4396-ab92-a869643ddb40-kube-api-access-4ggq8\") pod \"d361df08-ce57-4396-ab92-a869643ddb40\" (UID: \"d361df08-ce57-4396-ab92-a869643ddb40\") "
Sep 11 12:18:06.816954 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:06.816811    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d361df08-ce57-4396-ab92-a869643ddb40-istio-token\") pod \"d361df08-ce57-4396-ab92-a869643ddb40\" (UID: \"d361df08-ce57-4396-ab92-a869643ddb40\") "
Sep 11 12:18:06.816954 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:06.816853    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d361df08-ce57-4396-ab92-a869643ddb40-istiod-ca-cert\") pod \"d361df08-ce57-4396-ab92-a869643ddb40\" (UID: \"d361df08-ce57-4396-ab92-a869643ddb40\") "
Sep 11 12:18:06.816954 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:06.816898    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d361df08-ce57-4396-ab92-a869643ddb40-istio-envoy\") pod \"d361df08-ce57-4396-ab92-a869643ddb40\" (UID: \"d361df08-ce57-4396-ab92-a869643ddb40\") "
Sep 11 12:18:06.817300 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:06.816956    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d361df08-ce57-4396-ab92-a869643ddb40-workload-socket\") pod \"d361df08-ce57-4396-ab92-a869643ddb40\" (UID: \"d361df08-ce57-4396-ab92-a869643ddb40\") "
Sep 11 12:18:06.817300 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:06.816986    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d361df08-ce57-4396-ab92-a869643ddb40-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d361df08-ce57-4396-ab92-a869643ddb40" (UID: "d361df08-ce57-4396-ab92-a869643ddb40"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:18:06.817300 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:06.817012    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d361df08-ce57-4396-ab92-a869643ddb40-istio-data" (OuterVolumeSpecName: "istio-data") pod "d361df08-ce57-4396-ab92-a869643ddb40" (UID: "d361df08-ce57-4396-ab92-a869643ddb40"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:18:06.817300 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:06.817286    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d361df08-ce57-4396-ab92-a869643ddb40-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d361df08-ce57-4396-ab92-a869643ddb40" (UID: "d361df08-ce57-4396-ab92-a869643ddb40"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:18:06.817571 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:06.817350    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d361df08-ce57-4396-ab92-a869643ddb40-workload-certs\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:18:06.817571 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:06.817362    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d361df08-ce57-4396-ab92-a869643ddb40-istio-data\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:18:06.817571 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:06.817360    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d361df08-ce57-4396-ab92-a869643ddb40-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d361df08-ce57-4396-ab92-a869643ddb40" (UID: "d361df08-ce57-4396-ab92-a869643ddb40"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:18:06.817730 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:06.817561    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d361df08-ce57-4396-ab92-a869643ddb40-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d361df08-ce57-4396-ab92-a869643ddb40" (UID: "d361df08-ce57-4396-ab92-a869643ddb40"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:18:06.819152 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:06.819127    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d361df08-ce57-4396-ab92-a869643ddb40-kube-api-access-4ggq8" (OuterVolumeSpecName: "kube-api-access-4ggq8") pod "d361df08-ce57-4396-ab92-a869643ddb40" (UID: "d361df08-ce57-4396-ab92-a869643ddb40"). InnerVolumeSpecName "kube-api-access-4ggq8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:18:06.819152 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:06.819138    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d361df08-ce57-4396-ab92-a869643ddb40-istio-token" (OuterVolumeSpecName: "istio-token") pod "d361df08-ce57-4396-ab92-a869643ddb40" (UID: "d361df08-ce57-4396-ab92-a869643ddb40"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:18:06.819455 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:06.819432    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d361df08-ce57-4396-ab92-a869643ddb40-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d361df08-ce57-4396-ab92-a869643ddb40" (UID: "d361df08-ce57-4396-ab92-a869643ddb40"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 12:18:06.819455 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:06.819442    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d361df08-ce57-4396-ab92-a869643ddb40-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d361df08-ce57-4396-ab92-a869643ddb40" (UID: "d361df08-ce57-4396-ab92-a869643ddb40"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:18:06.918583 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:06.918544    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d361df08-ce57-4396-ab92-a869643ddb40-istio-podinfo\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:18:06.918583 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:06.918574    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d361df08-ce57-4396-ab92-a869643ddb40-credential-socket\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:18:06.918583 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:06.918584    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4ggq8\" (UniqueName: \"kubernetes.io/projected/d361df08-ce57-4396-ab92-a869643ddb40-kube-api-access-4ggq8\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:18:06.918583 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:06.918593    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d361df08-ce57-4396-ab92-a869643ddb40-istio-token\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:18:06.918863 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:06.918602    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d361df08-ce57-4396-ab92-a869643ddb40-istiod-ca-cert\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:18:06.918863 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:06.918609    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d361df08-ce57-4396-ab92-a869643ddb40-istio-envoy\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:18:06.918863 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:06.918617    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d361df08-ce57-4396-ab92-a869643ddb40-workload-socket\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:18:07.145622 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:07.145585    2579 generic.go:356] "Generic (PLEG): container finished" podID="d361df08-ce57-4396-ab92-a869643ddb40" containerID="c764a942ba0752066d3e2d5abdcbd085785f63a7bb4305412d18c2f0d5366d96" exitCode=0
Sep 11 12:18:07.145823 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:07.145650    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4"
Sep 11 12:18:07.145823 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:07.145670    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4" event={"ID":"d361df08-ce57-4396-ab92-a869643ddb40","Type":"ContainerDied","Data":"c764a942ba0752066d3e2d5abdcbd085785f63a7bb4305412d18c2f0d5366d96"}
Sep 11 12:18:07.145823 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:07.145764    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4" event={"ID":"d361df08-ce57-4396-ab92-a869643ddb40","Type":"ContainerDied","Data":"018520f9d42950254397ad26a3956439414a8b82e05e185c55aae177bbcb2212"}
Sep 11 12:18:07.145823 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:07.145788    2579 scope.go:117] "RemoveContainer" containerID="c764a942ba0752066d3e2d5abdcbd085785f63a7bb4305412d18c2f0d5366d96"
Sep 11 12:18:07.155678 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:07.155661    2579 scope.go:117] "RemoveContainer" containerID="79ede4f267ee1ca719c3113a9a26f78afe572addb1451dc20f70fb1722ed3dd0"
Sep 11 12:18:07.164082 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:07.164063    2579 scope.go:117] "RemoveContainer" containerID="c764a942ba0752066d3e2d5abdcbd085785f63a7bb4305412d18c2f0d5366d96"
Sep 11 12:18:07.164356 ip-10-0-142-145 kubenswrapper[2579]: E0911 12:18:07.164337    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c764a942ba0752066d3e2d5abdcbd085785f63a7bb4305412d18c2f0d5366d96\": container with ID starting with c764a942ba0752066d3e2d5abdcbd085785f63a7bb4305412d18c2f0d5366d96 not found: ID does not exist" containerID="c764a942ba0752066d3e2d5abdcbd085785f63a7bb4305412d18c2f0d5366d96"
Sep 11 12:18:07.164419 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:07.164363    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c764a942ba0752066d3e2d5abdcbd085785f63a7bb4305412d18c2f0d5366d96"} err="failed to get container status \"c764a942ba0752066d3e2d5abdcbd085785f63a7bb4305412d18c2f0d5366d96\": rpc error: code = NotFound desc = could not find container \"c764a942ba0752066d3e2d5abdcbd085785f63a7bb4305412d18c2f0d5366d96\": container with ID starting with c764a942ba0752066d3e2d5abdcbd085785f63a7bb4305412d18c2f0d5366d96 not found: ID does not exist"
Sep 11 12:18:07.164419 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:07.164377    2579 scope.go:117] "RemoveContainer" containerID="79ede4f267ee1ca719c3113a9a26f78afe572addb1451dc20f70fb1722ed3dd0"
Sep 11 12:18:07.164657 ip-10-0-142-145 kubenswrapper[2579]: E0911 12:18:07.164642    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79ede4f267ee1ca719c3113a9a26f78afe572addb1451dc20f70fb1722ed3dd0\": container with ID starting with 79ede4f267ee1ca719c3113a9a26f78afe572addb1451dc20f70fb1722ed3dd0 not found: ID does not exist" containerID="79ede4f267ee1ca719c3113a9a26f78afe572addb1451dc20f70fb1722ed3dd0"
Sep 11 12:18:07.164699 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:07.164662    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79ede4f267ee1ca719c3113a9a26f78afe572addb1451dc20f70fb1722ed3dd0"} err="failed to get container status \"79ede4f267ee1ca719c3113a9a26f78afe572addb1451dc20f70fb1722ed3dd0\": rpc error: code = NotFound desc = could not find container \"79ede4f267ee1ca719c3113a9a26f78afe572addb1451dc20f70fb1722ed3dd0\": container with ID starting with 79ede4f267ee1ca719c3113a9a26f78afe572addb1451dc20f70fb1722ed3dd0 not found: ID does not exist"
Sep 11 12:18:07.182795 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:07.182764    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4"]
Sep 11 12:18:07.188279 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:07.188250    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-bf9d5c-openshift-default-696d77f94d-p77v4"]
Sep 11 12:18:08.576661 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:08.576624    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d361df08-ce57-4396-ab92-a869643ddb40" path="/var/lib/kubelet/pods/d361df08-ce57-4396-ab92-a869643ddb40/volumes"
Sep 11 12:18:12.430536 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:12.430496    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-986cf2-29818815-tgqdt"]
Sep 11 12:18:12.433576 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:12.433544    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:18:12.437202 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:12.437177    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-986cf2-29818815-tgqdt"]
Sep 11 12:18:12.467950 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:12.467912    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-986cf2-65674c9d59-q6f47"]
Sep 11 12:18:12.468215 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:12.468178    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-986cf2-65674c9d59-q6f47" podUID="4e15f705-fc9a-4af7-b05a-5c359c81ac87" containerName="payload-processing" containerID="cri-o://35081373072a02f34a27321f26afdfc31e9b9df7980d39c01cd6f85568bd6f6c" gracePeriod=30
Sep 11 12:18:12.479863 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:12.478872    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-986cf2-57cd48794b-t8qcx"]
Sep 11 12:18:12.479863 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:12.479145    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-986cf2-57cd48794b-t8qcx" podUID="561d8ff4-6ec4-410a-bfae-409e687b916b" containerName="payload-pre-processing" containerID="cri-o://6dfa2af2f57b6e445b20d78e7b7cc9984222a72e1572d96d4a7935d42b145239" gracePeriod=30
Sep 11 12:18:12.525870 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:12.525833    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:18:12.578096 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:12.578056    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c0da0b7-c20a-450d-8811-3030a7197f07" path="/var/lib/kubelet/pods/3c0da0b7-c20a-450d-8811-3030a7197f07/volumes"
Sep 11 12:18:12.612904 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:12.612858    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:18:12.697062 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:12.697033    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-986cf2-h8pzf"]
Sep 11 12:18:12.700985 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:12.700951    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-986cf2-h8pzf"]
Sep 11 12:18:12.710339 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:12.710307    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:18:12.764018 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:12.763990    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-986cf2-65674c9d59-q6f47"
Sep 11 12:18:12.767472 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:12.767448    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-986cf2-57cd48794b-t8qcx"
Sep 11 12:18:12.806275 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:12.806235    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:18:12.878311 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:12.878275    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/561d8ff4-6ec4-410a-bfae-409e687b916b-plugins-config-volume\") pod \"561d8ff4-6ec4-410a-bfae-409e687b916b\" (UID: \"561d8ff4-6ec4-410a-bfae-409e687b916b\") "
Sep 11 12:18:12.878311 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:12.878319    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tpx2f\" (UniqueName: \"kubernetes.io/projected/4e15f705-fc9a-4af7-b05a-5c359c81ac87-kube-api-access-tpx2f\") pod \"4e15f705-fc9a-4af7-b05a-5c359c81ac87\" (UID: \"4e15f705-fc9a-4af7-b05a-5c359c81ac87\") "
Sep 11 12:18:12.878594 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:12.878417    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4lwsw\" (UniqueName: \"kubernetes.io/projected/561d8ff4-6ec4-410a-bfae-409e687b916b-kube-api-access-4lwsw\") pod \"561d8ff4-6ec4-410a-bfae-409e687b916b\" (UID: \"561d8ff4-6ec4-410a-bfae-409e687b916b\") "
Sep 11 12:18:12.878594 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:12.878471    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4e15f705-fc9a-4af7-b05a-5c359c81ac87-plugins-config-volume\") pod \"4e15f705-fc9a-4af7-b05a-5c359c81ac87\" (UID: \"4e15f705-fc9a-4af7-b05a-5c359c81ac87\") "
Sep 11 12:18:12.878794 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:12.878765    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/561d8ff4-6ec4-410a-bfae-409e687b916b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "561d8ff4-6ec4-410a-bfae-409e687b916b" (UID: "561d8ff4-6ec4-410a-bfae-409e687b916b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:18:12.878939 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:12.878912    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e15f705-fc9a-4af7-b05a-5c359c81ac87-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4e15f705-fc9a-4af7-b05a-5c359c81ac87" (UID: "4e15f705-fc9a-4af7-b05a-5c359c81ac87"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:18:12.880647 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:12.880613    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e15f705-fc9a-4af7-b05a-5c359c81ac87-kube-api-access-tpx2f" (OuterVolumeSpecName: "kube-api-access-tpx2f") pod "4e15f705-fc9a-4af7-b05a-5c359c81ac87" (UID: "4e15f705-fc9a-4af7-b05a-5c359c81ac87"). InnerVolumeSpecName "kube-api-access-tpx2f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:18:12.880748 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:12.880706    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/561d8ff4-6ec4-410a-bfae-409e687b916b-kube-api-access-4lwsw" (OuterVolumeSpecName: "kube-api-access-4lwsw") pod "561d8ff4-6ec4-410a-bfae-409e687b916b" (UID: "561d8ff4-6ec4-410a-bfae-409e687b916b"). InnerVolumeSpecName "kube-api-access-4lwsw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:18:12.979302 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:12.979216    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4lwsw\" (UniqueName: \"kubernetes.io/projected/561d8ff4-6ec4-410a-bfae-409e687b916b-kube-api-access-4lwsw\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:18:12.979302 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:12.979250    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4e15f705-fc9a-4af7-b05a-5c359c81ac87-plugins-config-volume\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:18:12.979302 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:12.979259    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/561d8ff4-6ec4-410a-bfae-409e687b916b-plugins-config-volume\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:18:12.979302 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:12.979268    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tpx2f\" (UniqueName: \"kubernetes.io/projected/4e15f705-fc9a-4af7-b05a-5c359c81ac87-kube-api-access-tpx2f\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:18:13.047038 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:13.047002    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:18:13.180872 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:13.180838    2579 generic.go:356] "Generic (PLEG): container finished" podID="4e15f705-fc9a-4af7-b05a-5c359c81ac87" containerID="35081373072a02f34a27321f26afdfc31e9b9df7980d39c01cd6f85568bd6f6c" exitCode=0
Sep 11 12:18:13.181050 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:13.180885    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-986cf2-65674c9d59-q6f47"
Sep 11 12:18:13.181050 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:13.180926    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-986cf2-65674c9d59-q6f47" event={"ID":"4e15f705-fc9a-4af7-b05a-5c359c81ac87","Type":"ContainerDied","Data":"35081373072a02f34a27321f26afdfc31e9b9df7980d39c01cd6f85568bd6f6c"}
Sep 11 12:18:13.181050 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:13.180981    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-986cf2-65674c9d59-q6f47" event={"ID":"4e15f705-fc9a-4af7-b05a-5c359c81ac87","Type":"ContainerDied","Data":"ef854a1046d995164d38c131e0183cc7cfee239eefc2b4afc607bc4787c6393d"}
Sep 11 12:18:13.181050 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:13.181009    2579 scope.go:117] "RemoveContainer" containerID="35081373072a02f34a27321f26afdfc31e9b9df7980d39c01cd6f85568bd6f6c"
Sep 11 12:18:13.182690 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:13.182655    2579 generic.go:356] "Generic (PLEG): container finished" podID="561d8ff4-6ec4-410a-bfae-409e687b916b" containerID="6dfa2af2f57b6e445b20d78e7b7cc9984222a72e1572d96d4a7935d42b145239" exitCode=0
Sep 11 12:18:13.182803 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:13.182739    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-986cf2-57cd48794b-t8qcx"
Sep 11 12:18:13.182866 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:13.182736    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-986cf2-57cd48794b-t8qcx" event={"ID":"561d8ff4-6ec4-410a-bfae-409e687b916b","Type":"ContainerDied","Data":"6dfa2af2f57b6e445b20d78e7b7cc9984222a72e1572d96d4a7935d42b145239"}
Sep 11 12:18:13.182929 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:13.182858    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-986cf2-57cd48794b-t8qcx" event={"ID":"561d8ff4-6ec4-410a-bfae-409e687b916b","Type":"ContainerDied","Data":"b4c58dfcf92c7664c77d11b5bfd82d0666f99c878a12a31ba590f5b642068290"}
Sep 11 12:18:13.191622 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:13.191604    2579 scope.go:117] "RemoveContainer" containerID="35081373072a02f34a27321f26afdfc31e9b9df7980d39c01cd6f85568bd6f6c"
Sep 11 12:18:13.191927 ip-10-0-142-145 kubenswrapper[2579]: E0911 12:18:13.191905    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35081373072a02f34a27321f26afdfc31e9b9df7980d39c01cd6f85568bd6f6c\": container with ID starting with 35081373072a02f34a27321f26afdfc31e9b9df7980d39c01cd6f85568bd6f6c not found: ID does not exist" containerID="35081373072a02f34a27321f26afdfc31e9b9df7980d39c01cd6f85568bd6f6c"
Sep 11 12:18:13.192015 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:13.191933    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35081373072a02f34a27321f26afdfc31e9b9df7980d39c01cd6f85568bd6f6c"} err="failed to get container status \"35081373072a02f34a27321f26afdfc31e9b9df7980d39c01cd6f85568bd6f6c\": rpc error: code = NotFound desc = could not find container \"35081373072a02f34a27321f26afdfc31e9b9df7980d39c01cd6f85568bd6f6c\": container with ID starting with 35081373072a02f34a27321f26afdfc31e9b9df7980d39c01cd6f85568bd6f6c not found: ID does not exist"
Sep 11 12:18:13.192015 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:13.191950    2579 scope.go:117] "RemoveContainer" containerID="6dfa2af2f57b6e445b20d78e7b7cc9984222a72e1572d96d4a7935d42b145239"
Sep 11 12:18:13.200694 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:13.200675    2579 scope.go:117] "RemoveContainer" containerID="6dfa2af2f57b6e445b20d78e7b7cc9984222a72e1572d96d4a7935d42b145239"
Sep 11 12:18:13.201004 ip-10-0-142-145 kubenswrapper[2579]: E0911 12:18:13.200985    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6dfa2af2f57b6e445b20d78e7b7cc9984222a72e1572d96d4a7935d42b145239\": container with ID starting with 6dfa2af2f57b6e445b20d78e7b7cc9984222a72e1572d96d4a7935d42b145239 not found: ID does not exist" containerID="6dfa2af2f57b6e445b20d78e7b7cc9984222a72e1572d96d4a7935d42b145239"
Sep 11 12:18:13.201055 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:13.201011    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6dfa2af2f57b6e445b20d78e7b7cc9984222a72e1572d96d4a7935d42b145239"} err="failed to get container status \"6dfa2af2f57b6e445b20d78e7b7cc9984222a72e1572d96d4a7935d42b145239\": rpc error: code = NotFound desc = could not find container \"6dfa2af2f57b6e445b20d78e7b7cc9984222a72e1572d96d4a7935d42b145239\": container with ID starting with 6dfa2af2f57b6e445b20d78e7b7cc9984222a72e1572d96d4a7935d42b145239 not found: ID does not exist"
Sep 11 12:18:13.208969 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:13.208948    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-986cf2-57cd48794b-t8qcx"]
Sep 11 12:18:13.213118 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:13.213096    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-986cf2-57cd48794b-t8qcx"]
Sep 11 12:18:13.223814 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:13.223792    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-986cf2-65674c9d59-q6f47"]
Sep 11 12:18:13.227613 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:13.227590    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-986cf2-65674c9d59-q6f47"]
Sep 11 12:18:13.419075 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:13.419046    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:18:14.576367 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:14.576332    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3205c4be-9d94-4e23-84c1-99c56e0e38cb" path="/var/lib/kubelet/pods/3205c4be-9d94-4e23-84c1-99c56e0e38cb/volumes"
Sep 11 12:18:14.576771 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:14.576727    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e15f705-fc9a-4af7-b05a-5c359c81ac87" path="/var/lib/kubelet/pods/4e15f705-fc9a-4af7-b05a-5c359c81ac87/volumes"
Sep 11 12:18:14.577037 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:14.577024    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="561d8ff4-6ec4-410a-bfae-409e687b916b" path="/var/lib/kubelet/pods/561d8ff4-6ec4-410a-bfae-409e687b916b/volumes"
Sep 11 12:18:15.976175 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:15.976139    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:18:16.649066 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:16.649037    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-ch96n_1f0f9fa5-cbd1-468b-ae81-82b48c101d37/console-operator/2.log"
Sep 11 12:18:16.651540 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:16.651520    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-ch96n_1f0f9fa5-cbd1-468b-ae81-82b48c101d37/console-operator/2.log"
Sep 11 12:18:19.235859 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:19.235823    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:18:19.335044 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:19.335001    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:18:19.523955 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:19.523859    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:18:19.893313 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:19.893278    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-af081c-4vgjg"]
Sep 11 12:18:19.893704 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:19.893691    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d361df08-ce57-4396-ab92-a869643ddb40" containerName="istio-proxy"
Sep 11 12:18:19.893767 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:19.893707    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d361df08-ce57-4396-ab92-a869643ddb40" containerName="istio-proxy"
Sep 11 12:18:19.893767 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:19.893720    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e15f705-fc9a-4af7-b05a-5c359c81ac87" containerName="payload-processing"
Sep 11 12:18:19.893767 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:19.893726    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e15f705-fc9a-4af7-b05a-5c359c81ac87" containerName="payload-processing"
Sep 11 12:18:19.893767 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:19.893735    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="561d8ff4-6ec4-410a-bfae-409e687b916b" containerName="payload-pre-processing"
Sep 11 12:18:19.893767 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:19.893741    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="561d8ff4-6ec4-410a-bfae-409e687b916b" containerName="payload-pre-processing"
Sep 11 12:18:19.893767 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:19.893748    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d361df08-ce57-4396-ab92-a869643ddb40" containerName="istio-proxy"
Sep 11 12:18:19.893767 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:19.893754    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d361df08-ce57-4396-ab92-a869643ddb40" containerName="istio-proxy"
Sep 11 12:18:19.893978 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:19.893774    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3205c4be-9d94-4e23-84c1-99c56e0e38cb" containerName="revoke-keys"
Sep 11 12:18:19.893978 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:19.893779    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="3205c4be-9d94-4e23-84c1-99c56e0e38cb" containerName="revoke-keys"
Sep 11 12:18:19.893978 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:19.893837    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="3205c4be-9d94-4e23-84c1-99c56e0e38cb" containerName="revoke-keys"
Sep 11 12:18:19.893978 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:19.893846    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d361df08-ce57-4396-ab92-a869643ddb40" containerName="istio-proxy"
Sep 11 12:18:19.893978 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:19.893854    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="561d8ff4-6ec4-410a-bfae-409e687b916b" containerName="payload-pre-processing"
Sep 11 12:18:19.893978 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:19.893861    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e15f705-fc9a-4af7-b05a-5c359c81ac87" containerName="payload-processing"
Sep 11 12:18:19.897099 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:19.897083    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-af081c-4vgjg"
Sep 11 12:18:19.899992 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:19.899969    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 12:18:19.900115 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:19.900052    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9xkpf\""
Sep 11 12:18:19.906915 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:19.906888    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-af081c-4vgjg"]
Sep 11 12:18:19.991777 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:19.991741    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:18:20.047105 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:20.047058    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/40b87077-6856-48fd-8a56-f107d7f84ad7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-af081c-4vgjg\" (UID: \"40b87077-6856-48fd-8a56-f107d7f84ad7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-af081c-4vgjg"
Sep 11 12:18:20.148058 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:20.147973    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/40b87077-6856-48fd-8a56-f107d7f84ad7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-af081c-4vgjg\" (UID: \"40b87077-6856-48fd-8a56-f107d7f84ad7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-af081c-4vgjg"
Sep 11 12:18:20.148611 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:20.148593    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/40b87077-6856-48fd-8a56-f107d7f84ad7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-af081c-4vgjg\" (UID: \"40b87077-6856-48fd-8a56-f107d7f84ad7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-af081c-4vgjg"
Sep 11 12:18:20.209298 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:20.209265    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-af081c-4vgjg"
Sep 11 12:18:20.488175 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:20.488101    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-af081c-4vgjg"]
Sep 11 12:18:20.491910 ip-10-0-142-145 kubenswrapper[2579]: W0911 12:18:20.491873    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40b87077_6856_48fd_8a56_f107d7f84ad7.slice/crio-a0c7543a0b0aa6282bdb9229583e52e5e585a515ac0326d2d6f8ab6fe641c0a1 WatchSource:0}: Error finding container a0c7543a0b0aa6282bdb9229583e52e5e585a515ac0326d2d6f8ab6fe641c0a1: Status 404 returned error can't find the container with id a0c7543a0b0aa6282bdb9229583e52e5e585a515ac0326d2d6f8ab6fe641c0a1
Sep 11 12:18:21.234043 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:21.234006    2579 generic.go:356] "Generic (PLEG): container finished" podID="40b87077-6856-48fd-8a56-f107d7f84ad7" containerID="712b37554163be72f274a047bce54768bdbf503ced53b5536d34a17141a17061" exitCode=0
Sep 11 12:18:21.234212 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:21.234091    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-af081c-4vgjg" event={"ID":"40b87077-6856-48fd-8a56-f107d7f84ad7","Type":"ContainerDied","Data":"712b37554163be72f274a047bce54768bdbf503ced53b5536d34a17141a17061"}
Sep 11 12:18:21.234212 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:21.234150    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-af081c-4vgjg" event={"ID":"40b87077-6856-48fd-8a56-f107d7f84ad7","Type":"ContainerStarted","Data":"a0c7543a0b0aa6282bdb9229583e52e5e585a515ac0326d2d6f8ab6fe641c0a1"}
Sep 11 12:18:22.377918 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:22.377893    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-af081c-4vgjg"
Sep 11 12:18:22.469685 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:22.469650    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/40b87077-6856-48fd-8a56-f107d7f84ad7-maas-api-service-ca\") pod \"40b87077-6856-48fd-8a56-f107d7f84ad7\" (UID: \"40b87077-6856-48fd-8a56-f107d7f84ad7\") "
Sep 11 12:18:22.470025 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:22.470002    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40b87077-6856-48fd-8a56-f107d7f84ad7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "40b87077-6856-48fd-8a56-f107d7f84ad7" (UID: "40b87077-6856-48fd-8a56-f107d7f84ad7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:18:22.570656 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:22.570564    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/40b87077-6856-48fd-8a56-f107d7f84ad7-maas-api-service-ca\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:18:23.246747 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:23.246714    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-af081c-4vgjg"
Sep 11 12:18:23.246937 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:23.246749    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-af081c-4vgjg" event={"ID":"40b87077-6856-48fd-8a56-f107d7f84ad7","Type":"ContainerDied","Data":"a0c7543a0b0aa6282bdb9229583e52e5e585a515ac0326d2d6f8ab6fe641c0a1"}
Sep 11 12:18:23.246937 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:23.246804    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a0c7543a0b0aa6282bdb9229583e52e5e585a515ac0326d2d6f8ab6fe641c0a1"
Sep 11 12:18:30.641418 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:30.641365    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:18:30.676305 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:30.676271    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-af081c-56499d888b-pg28q"]
Sep 11 12:18:30.676546 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:30.676518    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-af081c-56499d888b-pg28q" podUID="8c9771a3-444a-4013-b989-10ab1e733615" containerName="payload-pre-processing" containerID="cri-o://3ccbb68e189052dee1600d07f429df1a656853d60d789a7ff3e854aa83d03576" gracePeriod=30
Sep 11 12:18:30.741357 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:30.741321    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:18:30.840716 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:30.840679    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:18:30.849200 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:30.849163    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-af081c-4vgjg"]
Sep 11 12:18:30.852963 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:30.852933    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-af081c-4vgjg"]
Sep 11 12:18:30.938723 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:30.938695    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-af081c-56499d888b-pg28q"
Sep 11 12:18:30.943035 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:30.942995    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:18:31.054259 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:31.054223    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c9771a3-444a-4013-b989-10ab1e733615-plugins-config-volume\") pod \"8c9771a3-444a-4013-b989-10ab1e733615\" (UID: \"8c9771a3-444a-4013-b989-10ab1e733615\") "
Sep 11 12:18:31.054449 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:31.054326    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k6l2m\" (UniqueName: \"kubernetes.io/projected/8c9771a3-444a-4013-b989-10ab1e733615-kube-api-access-k6l2m\") pod \"8c9771a3-444a-4013-b989-10ab1e733615\" (UID: \"8c9771a3-444a-4013-b989-10ab1e733615\") "
Sep 11 12:18:31.054761 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:31.054733    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c9771a3-444a-4013-b989-10ab1e733615-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8c9771a3-444a-4013-b989-10ab1e733615" (UID: "8c9771a3-444a-4013-b989-10ab1e733615"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:18:31.056554 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:31.056534    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c9771a3-444a-4013-b989-10ab1e733615-kube-api-access-k6l2m" (OuterVolumeSpecName: "kube-api-access-k6l2m") pod "8c9771a3-444a-4013-b989-10ab1e733615" (UID: "8c9771a3-444a-4013-b989-10ab1e733615"). InnerVolumeSpecName "kube-api-access-k6l2m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:18:31.155329 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:31.155291    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k6l2m\" (UniqueName: \"kubernetes.io/projected/8c9771a3-444a-4013-b989-10ab1e733615-kube-api-access-k6l2m\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:18:31.155329 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:31.155324    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c9771a3-444a-4013-b989-10ab1e733615-plugins-config-volume\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:18:31.233328 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:31.233195    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:18:31.292297 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:31.292256    2579 generic.go:356] "Generic (PLEG): container finished" podID="8c9771a3-444a-4013-b989-10ab1e733615" containerID="3ccbb68e189052dee1600d07f429df1a656853d60d789a7ff3e854aa83d03576" exitCode=0
Sep 11 12:18:31.292497 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:31.292306    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-af081c-56499d888b-pg28q"
Sep 11 12:18:31.292497 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:31.292342    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-af081c-56499d888b-pg28q" event={"ID":"8c9771a3-444a-4013-b989-10ab1e733615","Type":"ContainerDied","Data":"3ccbb68e189052dee1600d07f429df1a656853d60d789a7ff3e854aa83d03576"}
Sep 11 12:18:31.292497 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:31.292425    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-af081c-56499d888b-pg28q" event={"ID":"8c9771a3-444a-4013-b989-10ab1e733615","Type":"ContainerDied","Data":"deefaa25f2b52207077f4a409581820bbba2f944af9ede30944c222d827b96cb"}
Sep 11 12:18:31.292497 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:31.292452    2579 scope.go:117] "RemoveContainer" containerID="3ccbb68e189052dee1600d07f429df1a656853d60d789a7ff3e854aa83d03576"
Sep 11 12:18:31.301931 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:31.301911    2579 scope.go:117] "RemoveContainer" containerID="3ccbb68e189052dee1600d07f429df1a656853d60d789a7ff3e854aa83d03576"
Sep 11 12:18:31.302237 ip-10-0-142-145 kubenswrapper[2579]: E0911 12:18:31.302218    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ccbb68e189052dee1600d07f429df1a656853d60d789a7ff3e854aa83d03576\": container with ID starting with 3ccbb68e189052dee1600d07f429df1a656853d60d789a7ff3e854aa83d03576 not found: ID does not exist" containerID="3ccbb68e189052dee1600d07f429df1a656853d60d789a7ff3e854aa83d03576"
Sep 11 12:18:31.302285 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:31.302247    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ccbb68e189052dee1600d07f429df1a656853d60d789a7ff3e854aa83d03576"} err="failed to get container status \"3ccbb68e189052dee1600d07f429df1a656853d60d789a7ff3e854aa83d03576\": rpc error: code = NotFound desc = could not find container \"3ccbb68e189052dee1600d07f429df1a656853d60d789a7ff3e854aa83d03576\": container with ID starting with 3ccbb68e189052dee1600d07f429df1a656853d60d789a7ff3e854aa83d03576 not found: ID does not exist"
Sep 11 12:18:31.314847 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:31.314819    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-af081c-56499d888b-pg28q"]
Sep 11 12:18:31.319217 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:31.319190    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-af081c-56499d888b-pg28q"]
Sep 11 12:18:32.575921 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:32.575890    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40b87077-6856-48fd-8a56-f107d7f84ad7" path="/var/lib/kubelet/pods/40b87077-6856-48fd-8a56-f107d7f84ad7/volumes"
Sep 11 12:18:32.576311 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:32.576242    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c9771a3-444a-4013-b989-10ab1e733615" path="/var/lib/kubelet/pods/8c9771a3-444a-4013-b989-10ab1e733615/volumes"
Sep 11 12:18:33.559109 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:33.559072    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:18:37.154871 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:37.154833    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:18:37.256011 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:37.255966    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:18:37.498405 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:37.498301    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:18:37.779517 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:37.779433    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-d844cc-p2s5p"]
Sep 11 12:18:37.779847 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:37.779834    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40b87077-6856-48fd-8a56-f107d7f84ad7" containerName="revoke-keys"
Sep 11 12:18:37.779892 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:37.779848    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="40b87077-6856-48fd-8a56-f107d7f84ad7" containerName="revoke-keys"
Sep 11 12:18:37.779892 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:37.779861    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c9771a3-444a-4013-b989-10ab1e733615" containerName="payload-pre-processing"
Sep 11 12:18:37.779892 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:37.779866    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c9771a3-444a-4013-b989-10ab1e733615" containerName="payload-pre-processing"
Sep 11 12:18:37.779990 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:37.779944    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d361df08-ce57-4396-ab92-a869643ddb40" containerName="istio-proxy"
Sep 11 12:18:37.779990 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:37.779951    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="40b87077-6856-48fd-8a56-f107d7f84ad7" containerName="revoke-keys"
Sep 11 12:18:37.779990 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:37.779959    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c9771a3-444a-4013-b989-10ab1e733615" containerName="payload-pre-processing"
Sep 11 12:18:37.784491 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:37.784474    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-d844cc-p2s5p"
Sep 11 12:18:37.787015 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:37.786985    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 12:18:37.787175 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:37.787033    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9xkpf\""
Sep 11 12:18:37.791274 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:37.791246    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-d844cc-p2s5p"]
Sep 11 12:18:37.887906 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:37.887874    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:18:37.919203 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:37.919161    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b534c1cc-7ec6-43d4-b7fa-f2f3ed931a6b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-d844cc-p2s5p\" (UID: \"b534c1cc-7ec6-43d4-b7fa-f2f3ed931a6b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-d844cc-p2s5p"
Sep 11 12:18:38.020575 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:38.020541    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b534c1cc-7ec6-43d4-b7fa-f2f3ed931a6b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-d844cc-p2s5p\" (UID: \"b534c1cc-7ec6-43d4-b7fa-f2f3ed931a6b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-d844cc-p2s5p"
Sep 11 12:18:38.021164 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:38.021145    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b534c1cc-7ec6-43d4-b7fa-f2f3ed931a6b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-d844cc-p2s5p\" (UID: \"b534c1cc-7ec6-43d4-b7fa-f2f3ed931a6b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-d844cc-p2s5p"
Sep 11 12:18:38.096860 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:38.096818    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-d844cc-p2s5p"
Sep 11 12:18:38.374309 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:38.374279    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-d844cc-p2s5p"]
Sep 11 12:18:38.376832 ip-10-0-142-145 kubenswrapper[2579]: W0911 12:18:38.376797    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb534c1cc_7ec6_43d4_b7fa_f2f3ed931a6b.slice/crio-1c82f31ae822804d0f9615bfd1df683c8e97d6b2f3ba9382df25cce46a3d198d WatchSource:0}: Error finding container 1c82f31ae822804d0f9615bfd1df683c8e97d6b2f3ba9382df25cce46a3d198d: Status 404 returned error can't find the container with id 1c82f31ae822804d0f9615bfd1df683c8e97d6b2f3ba9382df25cce46a3d198d
Sep 11 12:18:39.338438 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:39.338329    2579 generic.go:356] "Generic (PLEG): container finished" podID="b534c1cc-7ec6-43d4-b7fa-f2f3ed931a6b" containerID="9f53ee31ef3b94851b23762d1e92b4c6b15a1a922f9013ccb05a1447772dbd9e" exitCode=0
Sep 11 12:18:39.338438 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:39.338418    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-d844cc-p2s5p" event={"ID":"b534c1cc-7ec6-43d4-b7fa-f2f3ed931a6b","Type":"ContainerDied","Data":"9f53ee31ef3b94851b23762d1e92b4c6b15a1a922f9013ccb05a1447772dbd9e"}
Sep 11 12:18:39.338633 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:39.338469    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-d844cc-p2s5p" event={"ID":"b534c1cc-7ec6-43d4-b7fa-f2f3ed931a6b","Type":"ContainerStarted","Data":"1c82f31ae822804d0f9615bfd1df683c8e97d6b2f3ba9382df25cce46a3d198d"}
Sep 11 12:18:40.487796 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:40.487768    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-d844cc-p2s5p"
Sep 11 12:18:40.541245 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:40.541209    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b534c1cc-7ec6-43d4-b7fa-f2f3ed931a6b-maas-api-service-ca\") pod \"b534c1cc-7ec6-43d4-b7fa-f2f3ed931a6b\" (UID: \"b534c1cc-7ec6-43d4-b7fa-f2f3ed931a6b\") "
Sep 11 12:18:40.541664 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:40.541640    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b534c1cc-7ec6-43d4-b7fa-f2f3ed931a6b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b534c1cc-7ec6-43d4-b7fa-f2f3ed931a6b" (UID: "b534c1cc-7ec6-43d4-b7fa-f2f3ed931a6b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:18:40.642280 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:40.642187    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b534c1cc-7ec6-43d4-b7fa-f2f3ed931a6b-maas-api-service-ca\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:18:41.357109 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:41.357082    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-d844cc-p2s5p"
Sep 11 12:18:41.357109 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:41.357085    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-d844cc-p2s5p" event={"ID":"b534c1cc-7ec6-43d4-b7fa-f2f3ed931a6b","Type":"ContainerDied","Data":"1c82f31ae822804d0f9615bfd1df683c8e97d6b2f3ba9382df25cce46a3d198d"}
Sep 11 12:18:41.357296 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:41.357133    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1c82f31ae822804d0f9615bfd1df683c8e97d6b2f3ba9382df25cce46a3d198d"
Sep 11 12:18:48.445470 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:48.445367    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:18:48.455793 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:48.455763    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-d844cc-29818815-29fh8"]
Sep 11 12:18:48.461899 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:48.461871    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-d844cc-29818815-29fh8"]
Sep 11 12:18:48.490533 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:48.490496    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-d844cc-6c8446bb85-frh5c"]
Sep 11 12:18:48.490729 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:48.490706    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-d844cc-6c8446bb85-frh5c" podUID="fa552843-e4cf-447a-a25d-288eb7f9660a" containerName="payload-processing" containerID="cri-o://999122a4e67477dfc122d9223f0f9c59e4edf90de5f1bd16001644bdb963768e" gracePeriod=30
Sep 11 12:18:48.546110 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:48.546075    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:18:48.577144 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:48.577109    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5bf9290f-b75a-46ea-9d49-c838966cda75" path="/var/lib/kubelet/pods/5bf9290f-b75a-46ea-9d49-c838966cda75/volumes"
Sep 11 12:18:48.631639 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:48.631601    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:18:48.732075 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:48.732045    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:18:48.750579 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:48.750555    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-d844cc-6c8446bb85-frh5c"
Sep 11 12:18:48.762170 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:48.762145    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-d844cc-p2s5p"]
Sep 11 12:18:48.765308 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:48.765283    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-d844cc-p2s5p"]
Sep 11 12:18:48.819826 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:48.819793    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa552843-e4cf-447a-a25d-288eb7f9660a-plugins-config-volume\") pod \"fa552843-e4cf-447a-a25d-288eb7f9660a\" (UID: \"fa552843-e4cf-447a-a25d-288eb7f9660a\") "
Sep 11 12:18:48.820012 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:48.819912    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ntq9s\" (UniqueName: \"kubernetes.io/projected/fa552843-e4cf-447a-a25d-288eb7f9660a-kube-api-access-ntq9s\") pod \"fa552843-e4cf-447a-a25d-288eb7f9660a\" (UID: \"fa552843-e4cf-447a-a25d-288eb7f9660a\") "
Sep 11 12:18:48.820256 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:48.820225    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa552843-e4cf-447a-a25d-288eb7f9660a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fa552843-e4cf-447a-a25d-288eb7f9660a" (UID: "fa552843-e4cf-447a-a25d-288eb7f9660a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:18:48.822042 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:48.822019    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa552843-e4cf-447a-a25d-288eb7f9660a-kube-api-access-ntq9s" (OuterVolumeSpecName: "kube-api-access-ntq9s") pod "fa552843-e4cf-447a-a25d-288eb7f9660a" (UID: "fa552843-e4cf-447a-a25d-288eb7f9660a"). InnerVolumeSpecName "kube-api-access-ntq9s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:18:48.839388 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:48.839358    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:18:48.921510 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:48.921474    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ntq9s\" (UniqueName: \"kubernetes.io/projected/fa552843-e4cf-447a-a25d-288eb7f9660a-kube-api-access-ntq9s\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:18:48.921510 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:48.921504    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa552843-e4cf-447a-a25d-288eb7f9660a-plugins-config-volume\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:18:49.154619 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:49.154582    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:18:49.407542 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:49.407454    2579 generic.go:356] "Generic (PLEG): container finished" podID="fa552843-e4cf-447a-a25d-288eb7f9660a" containerID="999122a4e67477dfc122d9223f0f9c59e4edf90de5f1bd16001644bdb963768e" exitCode=0
Sep 11 12:18:49.407542 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:49.407506    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-d844cc-6c8446bb85-frh5c"
Sep 11 12:18:49.407734 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:49.407541    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-d844cc-6c8446bb85-frh5c" event={"ID":"fa552843-e4cf-447a-a25d-288eb7f9660a","Type":"ContainerDied","Data":"999122a4e67477dfc122d9223f0f9c59e4edf90de5f1bd16001644bdb963768e"}
Sep 11 12:18:49.407734 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:49.407594    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-d844cc-6c8446bb85-frh5c" event={"ID":"fa552843-e4cf-447a-a25d-288eb7f9660a","Type":"ContainerDied","Data":"a991b0e0427dc502e367aaa4c06d0c8802214b07184d8217297660f391f7bfd9"}
Sep 11 12:18:49.407734 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:49.407616    2579 scope.go:117] "RemoveContainer" containerID="999122a4e67477dfc122d9223f0f9c59e4edf90de5f1bd16001644bdb963768e"
Sep 11 12:18:49.417117 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:49.417099    2579 scope.go:117] "RemoveContainer" containerID="999122a4e67477dfc122d9223f0f9c59e4edf90de5f1bd16001644bdb963768e"
Sep 11 12:18:49.417419 ip-10-0-142-145 kubenswrapper[2579]: E0911 12:18:49.417402    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"999122a4e67477dfc122d9223f0f9c59e4edf90de5f1bd16001644bdb963768e\": container with ID starting with 999122a4e67477dfc122d9223f0f9c59e4edf90de5f1bd16001644bdb963768e not found: ID does not exist" containerID="999122a4e67477dfc122d9223f0f9c59e4edf90de5f1bd16001644bdb963768e"
Sep 11 12:18:49.417490 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:49.417431    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"999122a4e67477dfc122d9223f0f9c59e4edf90de5f1bd16001644bdb963768e"} err="failed to get container status \"999122a4e67477dfc122d9223f0f9c59e4edf90de5f1bd16001644bdb963768e\": rpc error: code = NotFound desc = could not find container \"999122a4e67477dfc122d9223f0f9c59e4edf90de5f1bd16001644bdb963768e\": container with ID starting with 999122a4e67477dfc122d9223f0f9c59e4edf90de5f1bd16001644bdb963768e not found: ID does not exist"
Sep 11 12:18:49.430969 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:49.430944    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-d844cc-6c8446bb85-frh5c"]
Sep 11 12:18:49.437432 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:49.437386    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-d844cc-6c8446bb85-frh5c"]
Sep 11 12:18:50.576488 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:50.576452    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b534c1cc-7ec6-43d4-b7fa-f2f3ed931a6b" path="/var/lib/kubelet/pods/b534c1cc-7ec6-43d4-b7fa-f2f3ed931a6b/volumes"
Sep 11 12:18:50.576876 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:50.576834    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa552843-e4cf-447a-a25d-288eb7f9660a" path="/var/lib/kubelet/pods/fa552843-e4cf-447a-a25d-288eb7f9660a/volumes"
Sep 11 12:18:51.271112 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:51.271077    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:18:55.819595 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:55.819561    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:18:56.157720 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:56.157685    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:18:56.213310 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:56.213270    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65"]
Sep 11 12:18:56.213545 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:56.213520    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65" podUID="45e86e88-5ccd-4c4c-8d31-8350e844f996" containerName="istio-proxy" containerID="cri-o://137cee4919336a89de681a2d292b81ce720e3d0fe244a7d20ec740435414c952" gracePeriod=30
Sep 11 12:18:56.425768 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:56.425679    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:18:56.514349 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:56.514314    2579 patch_prober.go:28] interesting pod/e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 11 12:18:56.514541 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:18:56.514367    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65" podUID="45e86e88-5ccd-4c4c-8d31-8350e844f996" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 12:19:00.168937 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:19:00.168903    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:19:01.476968 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:19:01.476900    2579 generic.go:356] "Generic (PLEG): container finished" podID="45e86e88-5ccd-4c4c-8d31-8350e844f996" containerID="137cee4919336a89de681a2d292b81ce720e3d0fe244a7d20ec740435414c952" exitCode=0
Sep 11 12:19:01.477286 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:19:01.476974    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65" event={"ID":"45e86e88-5ccd-4c4c-8d31-8350e844f996","Type":"ContainerDied","Data":"137cee4919336a89de681a2d292b81ce720e3d0fe244a7d20ec740435414c952"}
Sep 11 12:19:01.477286 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:19:01.477067    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65" event={"ID":"45e86e88-5ccd-4c4c-8d31-8350e844f996","Type":"ContainerDied","Data":"fd7fca3f9d12000bc1c3b8e2724b61d158768b802aca49235fab53d6f6f67410"}
Sep 11 12:19:01.477286 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:19:01.477085    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fd7fca3f9d12000bc1c3b8e2724b61d158768b802aca49235fab53d6f6f67410"
Sep 11 12:19:01.477286 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:19:01.477099    2579 scope.go:117] "RemoveContainer" containerID="8ddbae16db110bb06d24ae0afb63573f49c1581c4f9550ba5ccd2292f86c5c2c"
Sep 11 12:19:01.482708 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:19:01.482682    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65"
Sep 11 12:19:01.543890 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:19:01.543852    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/45e86e88-5ccd-4c4c-8d31-8350e844f996-workload-certs\") pod \"45e86e88-5ccd-4c4c-8d31-8350e844f996\" (UID: \"45e86e88-5ccd-4c4c-8d31-8350e844f996\") "
Sep 11 12:19:01.543890 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:19:01.543893    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/45e86e88-5ccd-4c4c-8d31-8350e844f996-istio-podinfo\") pod \"45e86e88-5ccd-4c4c-8d31-8350e844f996\" (UID: \"45e86e88-5ccd-4c4c-8d31-8350e844f996\") "
Sep 11 12:19:01.544128 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:19:01.543969    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/45e86e88-5ccd-4c4c-8d31-8350e844f996-istio-token\") pod \"45e86e88-5ccd-4c4c-8d31-8350e844f996\" (UID: \"45e86e88-5ccd-4c4c-8d31-8350e844f996\") "
Sep 11 12:19:01.544128 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:19:01.543993    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/45e86e88-5ccd-4c4c-8d31-8350e844f996-workload-socket\") pod \"45e86e88-5ccd-4c4c-8d31-8350e844f996\" (UID: \"45e86e88-5ccd-4c4c-8d31-8350e844f996\") "
Sep 11 12:19:01.544128 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:19:01.544031    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/45e86e88-5ccd-4c4c-8d31-8350e844f996-credential-socket\") pod \"45e86e88-5ccd-4c4c-8d31-8350e844f996\" (UID: \"45e86e88-5ccd-4c4c-8d31-8350e844f996\") "
Sep 11 12:19:01.544128 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:19:01.544057    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/45e86e88-5ccd-4c4c-8d31-8350e844f996-istiod-ca-cert\") pod \"45e86e88-5ccd-4c4c-8d31-8350e844f996\" (UID: \"45e86e88-5ccd-4c4c-8d31-8350e844f996\") "
Sep 11 12:19:01.544128 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:19:01.544081    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/45e86e88-5ccd-4c4c-8d31-8350e844f996-istio-envoy\") pod \"45e86e88-5ccd-4c4c-8d31-8350e844f996\" (UID: \"45e86e88-5ccd-4c4c-8d31-8350e844f996\") "
Sep 11 12:19:01.544128 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:19:01.544124    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4ptwz\" (UniqueName: \"kubernetes.io/projected/45e86e88-5ccd-4c4c-8d31-8350e844f996-kube-api-access-4ptwz\") pod \"45e86e88-5ccd-4c4c-8d31-8350e844f996\" (UID: \"45e86e88-5ccd-4c4c-8d31-8350e844f996\") "
Sep 11 12:19:01.544493 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:19:01.544123    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45e86e88-5ccd-4c4c-8d31-8350e844f996-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "45e86e88-5ccd-4c4c-8d31-8350e844f996" (UID: "45e86e88-5ccd-4c4c-8d31-8350e844f996"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:19:01.544493 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:19:01.544333    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45e86e88-5ccd-4c4c-8d31-8350e844f996-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "45e86e88-5ccd-4c4c-8d31-8350e844f996" (UID: "45e86e88-5ccd-4c4c-8d31-8350e844f996"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:19:01.544493 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:19:01.544383    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/45e86e88-5ccd-4c4c-8d31-8350e844f996-istio-data\") pod \"45e86e88-5ccd-4c4c-8d31-8350e844f996\" (UID: \"45e86e88-5ccd-4c4c-8d31-8350e844f996\") "
Sep 11 12:19:01.544493 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:19:01.544416    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45e86e88-5ccd-4c4c-8d31-8350e844f996-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "45e86e88-5ccd-4c4c-8d31-8350e844f996" (UID: "45e86e88-5ccd-4c4c-8d31-8350e844f996"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:19:01.544688 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:19:01.544541    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45e86e88-5ccd-4c4c-8d31-8350e844f996-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "45e86e88-5ccd-4c4c-8d31-8350e844f996" (UID: "45e86e88-5ccd-4c4c-8d31-8350e844f996"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:19:01.544752 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:19:01.544722    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45e86e88-5ccd-4c4c-8d31-8350e844f996-istio-data" (OuterVolumeSpecName: "istio-data") pod "45e86e88-5ccd-4c4c-8d31-8350e844f996" (UID: "45e86e88-5ccd-4c4c-8d31-8350e844f996"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:19:01.544903 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:19:01.544882    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/45e86e88-5ccd-4c4c-8d31-8350e844f996-workload-socket\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:19:01.544974 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:19:01.544908    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/45e86e88-5ccd-4c4c-8d31-8350e844f996-credential-socket\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:19:01.544974 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:19:01.544924    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/45e86e88-5ccd-4c4c-8d31-8350e844f996-istiod-ca-cert\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:19:01.544974 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:19:01.544938    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/45e86e88-5ccd-4c4c-8d31-8350e844f996-istio-data\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:19:01.544974 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:19:01.544952    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/45e86e88-5ccd-4c4c-8d31-8350e844f996-workload-certs\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:19:01.546273 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:19:01.546236    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45e86e88-5ccd-4c4c-8d31-8350e844f996-kube-api-access-4ptwz" (OuterVolumeSpecName: "kube-api-access-4ptwz") pod "45e86e88-5ccd-4c4c-8d31-8350e844f996" (UID: "45e86e88-5ccd-4c4c-8d31-8350e844f996"). InnerVolumeSpecName "kube-api-access-4ptwz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:19:01.546273 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:19:01.546245    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/45e86e88-5ccd-4c4c-8d31-8350e844f996-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "45e86e88-5ccd-4c4c-8d31-8350e844f996" (UID: "45e86e88-5ccd-4c4c-8d31-8350e844f996"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 12:19:01.546464 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:19:01.546369    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45e86e88-5ccd-4c4c-8d31-8350e844f996-istio-token" (OuterVolumeSpecName: "istio-token") pod "45e86e88-5ccd-4c4c-8d31-8350e844f996" (UID: "45e86e88-5ccd-4c4c-8d31-8350e844f996"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:19:01.546464 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:19:01.546409    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45e86e88-5ccd-4c4c-8d31-8350e844f996-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "45e86e88-5ccd-4c4c-8d31-8350e844f996" (UID: "45e86e88-5ccd-4c4c-8d31-8350e844f996"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:19:01.646510 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:19:01.646465    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/45e86e88-5ccd-4c4c-8d31-8350e844f996-istio-envoy\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:19:01.646510 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:19:01.646504    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4ptwz\" (UniqueName: \"kubernetes.io/projected/45e86e88-5ccd-4c4c-8d31-8350e844f996-kube-api-access-4ptwz\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:19:01.646510 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:19:01.646519    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/45e86e88-5ccd-4c4c-8d31-8350e844f996-istio-podinfo\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:19:01.646752 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:19:01.646528    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/45e86e88-5ccd-4c4c-8d31-8350e844f996-istio-token\") on node \"ip-10-0-142-145.ec2.internal\" DevicePath \"\""
Sep 11 12:19:02.484091 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:19:02.484060    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65"
Sep 11 12:19:02.548491 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:19:02.548460    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65"]
Sep 11 12:19:02.557227 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:19:02.557189    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-d844cc-openshift-default-d787c6bf4-zkk65"]
Sep 11 12:19:02.576544 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:19:02.576518    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="45e86e88-5ccd-4c4c-8d31-8350e844f996" path="/var/lib/kubelet/pods/45e86e88-5ccd-4c4c-8d31-8350e844f996/volumes"
Sep 11 12:19:16.852062 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:19:16.852030    2579 scope.go:117] "RemoveContainer" containerID="137cee4919336a89de681a2d292b81ce720e3d0fe244a7d20ec740435414c952"
Sep 11 12:19:23.238167 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:19:23.238130    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:20:16.881869 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:20:16.881834    2579 scope.go:117] "RemoveContainer" containerID="6618ab43795d33de65f27e4a9b1ce6cf845114816930da8e89ca922808f9ea65"
Sep 11 12:20:16.890370 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:20:16.890347    2579 scope.go:117] "RemoveContainer" containerID="bc17a6dd94034892bd2229cdf97a43bcef06e86a98418d1bccccc561e4bb4e5a"
Sep 11 12:20:16.899698 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:20:16.899673    2579 scope.go:117] "RemoveContainer" containerID="7bb0b2039a2c952ee09d7b3e99b21063751739e19ebf4139614f5600ebc0a13e"
Sep 11 12:20:21.415872 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:20:21.415826    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:20:21.971266 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:20:21.971226    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:20:22.373795 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:20:22.373762    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:20:23.722684 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:20:23.722636    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:20:47.690847 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:20:47.690809    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:20:56.186639 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:20:56.186603    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:20:59.095848 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:20:59.095812    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:21:01.537073 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:21:01.537034    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:21:03.977354 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:21:03.977318    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:21:05.077925 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:21:05.077882    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:21:16.928553 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:21:16.928464    2579 scope.go:117] "RemoveContainer" containerID="28174919688c0dca8e6be4c9e8d293768a6f685f533d3d388cc496ec00dbd857"
Sep 11 12:21:16.937243 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:21:16.937222    2579 scope.go:117] "RemoveContainer" containerID="97398cc2d2652c775caebebaa438d1b97372266292dff20de781a08c01ee0300"
Sep 11 12:21:16.945150 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:21:16.945127    2579 scope.go:117] "RemoveContainer" containerID="15a7870ebdfe2d67b1a132a8f33dbfa40dd440b9abb730dd9021881fd8b91a89"
Sep 11 12:21:16.952743 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:21:16.952722    2579 scope.go:117] "RemoveContainer" containerID="4ece65f372430b9bc7fdb0fc6d74752349d58c587b2d1d4a446b07de7f9b9bba"
Sep 11 12:21:18.786659 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:21:18.786611    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:21:26.675613 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:21:26.675578    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:21:55.646336 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:21:55.646295    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:21:55.756511 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:21:55.756470    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:22:11.239942 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:22:11.239905    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:22:11.342972 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:22:11.342929    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:22:16.473187 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:22:16.473148    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:22:16.585904 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:22:16.585865    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:22:56.084523 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:22:56.084478    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:23:02.959825 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:23:02.959782    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:23:10.462842 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:23:10.462801    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:23:16.691330 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:23:16.691303    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-ch96n_1f0f9fa5-cbd1-468b-ae81-82b48c101d37/console-operator/2.log"
Sep 11 12:23:16.693787 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:23:16.693762    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-ch96n_1f0f9fa5-cbd1-468b-ae81-82b48c101d37/console-operator/2.log"
Sep 11 12:23:16.989254 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:23:16.989169    2579 scope.go:117] "RemoveContainer" containerID="60a0a5aa8e567a29e2f654acb94d1f6a1ee7a149bc5bda8c6d8d0f52c294d1c4"
Sep 11 12:23:22.375962 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:23:22.375918    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:23:25.390569 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:23:25.390525    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:23:35.067642 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:23:35.067604    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:23:38.023816 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:23:38.023779    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:24:11.875613 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:24:11.875527    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:24:13.353067 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:24:13.353032    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:24:13.961327 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:24:13.961287    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:24:17.011362 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:24:17.011332    2579 scope.go:117] "RemoveContainer" containerID="f9ec99a04f2b6e3fe5099b0fd3a2f5780aa36cf0a7c4a0ca9ae731e0839bab3a"
Sep 11 12:24:23.268794 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:24:23.268757    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:24:32.470186 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:24:32.470146    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:24:32.785426 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:24:32.785329    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7b8f67d746-765t6"]
Sep 11 12:24:32.785993 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:24:32.785972    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa552843-e4cf-447a-a25d-288eb7f9660a" containerName="payload-processing"
Sep 11 12:24:32.786041 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:24:32.786000    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa552843-e4cf-447a-a25d-288eb7f9660a" containerName="payload-processing"
Sep 11 12:24:32.786041 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:24:32.786026    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45e86e88-5ccd-4c4c-8d31-8350e844f996" containerName="istio-proxy"
Sep 11 12:24:32.786041 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:24:32.786034    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="45e86e88-5ccd-4c4c-8d31-8350e844f996" containerName="istio-proxy"
Sep 11 12:24:32.786139 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:24:32.786052    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b534c1cc-7ec6-43d4-b7fa-f2f3ed931a6b" containerName="revoke-keys"
Sep 11 12:24:32.786139 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:24:32.786062    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b534c1cc-7ec6-43d4-b7fa-f2f3ed931a6b" containerName="revoke-keys"
Sep 11 12:24:32.786205 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:24:32.786185    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="45e86e88-5ccd-4c4c-8d31-8350e844f996" containerName="istio-proxy"
Sep 11 12:24:32.786205 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:24:32.786199    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="45e86e88-5ccd-4c4c-8d31-8350e844f996" containerName="istio-proxy"
Sep 11 12:24:32.786267 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:24:32.786210    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa552843-e4cf-447a-a25d-288eb7f9660a" containerName="payload-processing"
Sep 11 12:24:32.786267 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:24:32.786222    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="b534c1cc-7ec6-43d4-b7fa-f2f3ed931a6b" containerName="revoke-keys"
Sep 11 12:24:32.789833 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:24:32.789813    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7b8f67d746-765t6"
Sep 11 12:24:32.793826 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:24:32.793805    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 12:24:32.794133 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:24:32.793805    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-rrxjh\""
Sep 11 12:24:32.794321 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:24:32.794298    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 11 12:24:32.794500 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:24:32.793896    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 11 12:24:32.794606 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:24:32.793874    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 11 12:24:32.799905 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:24:32.799882    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7b8f67d746-765t6"]
Sep 11 12:24:32.837595 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:24:32.837560    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc0ee18c-f3fc-4dca-9d6f-59b1407f6eea-maas-api-tls\") pod \"maas-api-7b8f67d746-765t6\" (UID: \"dc0ee18c-f3fc-4dca-9d6f-59b1407f6eea\") " pod="odh-ai-gateway-infra/maas-api-7b8f67d746-765t6"
Sep 11 12:24:32.837768 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:24:32.837612    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc0ee18c-f3fc-4dca-9d6f-59b1407f6eea-openshift-service-ca\") pod \"maas-api-7b8f67d746-765t6\" (UID: \"dc0ee18c-f3fc-4dca-9d6f-59b1407f6eea\") " pod="odh-ai-gateway-infra/maas-api-7b8f67d746-765t6"
Sep 11 12:24:32.837768 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:24:32.837642    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc0ee18c-f3fc-4dca-9d6f-59b1407f6eea-metrics-certs\") pod \"maas-api-7b8f67d746-765t6\" (UID: \"dc0ee18c-f3fc-4dca-9d6f-59b1407f6eea\") " pod="odh-ai-gateway-infra/maas-api-7b8f67d746-765t6"
Sep 11 12:24:32.837768 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:24:32.837714    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbrm8\" (UniqueName: \"kubernetes.io/projected/dc0ee18c-f3fc-4dca-9d6f-59b1407f6eea-kube-api-access-tbrm8\") pod \"maas-api-7b8f67d746-765t6\" (UID: \"dc0ee18c-f3fc-4dca-9d6f-59b1407f6eea\") " pod="odh-ai-gateway-infra/maas-api-7b8f67d746-765t6"
Sep 11 12:24:32.939121 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:24:32.939088    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbrm8\" (UniqueName: \"kubernetes.io/projected/dc0ee18c-f3fc-4dca-9d6f-59b1407f6eea-kube-api-access-tbrm8\") pod \"maas-api-7b8f67d746-765t6\" (UID: \"dc0ee18c-f3fc-4dca-9d6f-59b1407f6eea\") " pod="odh-ai-gateway-infra/maas-api-7b8f67d746-765t6"
Sep 11 12:24:32.939321 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:24:32.939138    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc0ee18c-f3fc-4dca-9d6f-59b1407f6eea-maas-api-tls\") pod \"maas-api-7b8f67d746-765t6\" (UID: \"dc0ee18c-f3fc-4dca-9d6f-59b1407f6eea\") " pod="odh-ai-gateway-infra/maas-api-7b8f67d746-765t6"
Sep 11 12:24:32.939321 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:24:32.939172    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc0ee18c-f3fc-4dca-9d6f-59b1407f6eea-openshift-service-ca\") pod \"maas-api-7b8f67d746-765t6\" (UID: \"dc0ee18c-f3fc-4dca-9d6f-59b1407f6eea\") " pod="odh-ai-gateway-infra/maas-api-7b8f67d746-765t6"
Sep 11 12:24:32.939321 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:24:32.939291    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc0ee18c-f3fc-4dca-9d6f-59b1407f6eea-metrics-certs\") pod \"maas-api-7b8f67d746-765t6\" (UID: \"dc0ee18c-f3fc-4dca-9d6f-59b1407f6eea\") " pod="odh-ai-gateway-infra/maas-api-7b8f67d746-765t6"
Sep 11 12:24:32.939814 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:24:32.939792    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc0ee18c-f3fc-4dca-9d6f-59b1407f6eea-openshift-service-ca\") pod \"maas-api-7b8f67d746-765t6\" (UID: \"dc0ee18c-f3fc-4dca-9d6f-59b1407f6eea\") " pod="odh-ai-gateway-infra/maas-api-7b8f67d746-765t6"
Sep 11 12:24:32.941740 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:24:32.941702    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc0ee18c-f3fc-4dca-9d6f-59b1407f6eea-maas-api-tls\") pod \"maas-api-7b8f67d746-765t6\" (UID: \"dc0ee18c-f3fc-4dca-9d6f-59b1407f6eea\") " pod="odh-ai-gateway-infra/maas-api-7b8f67d746-765t6"
Sep 11 12:24:32.941740 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:24:32.941726    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc0ee18c-f3fc-4dca-9d6f-59b1407f6eea-metrics-certs\") pod \"maas-api-7b8f67d746-765t6\" (UID: \"dc0ee18c-f3fc-4dca-9d6f-59b1407f6eea\") " pod="odh-ai-gateway-infra/maas-api-7b8f67d746-765t6"
Sep 11 12:24:32.947299 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:24:32.947277    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbrm8\" (UniqueName: \"kubernetes.io/projected/dc0ee18c-f3fc-4dca-9d6f-59b1407f6eea-kube-api-access-tbrm8\") pod \"maas-api-7b8f67d746-765t6\" (UID: \"dc0ee18c-f3fc-4dca-9d6f-59b1407f6eea\") " pod="odh-ai-gateway-infra/maas-api-7b8f67d746-765t6"
Sep 11 12:24:33.102475 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:24:33.102435    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7b8f67d746-765t6"
Sep 11 12:24:33.396789 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:24:33.396757    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7b8f67d746-765t6"]
Sep 11 12:24:33.399740 ip-10-0-142-145 kubenswrapper[2579]: W0911 12:24:33.399712    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc0ee18c_f3fc_4dca_9d6f_59b1407f6eea.slice/crio-f52994bb928b4a0154e332ed5af0c646c189f82678d5c15d3b1550ca0383645b WatchSource:0}: Error finding container f52994bb928b4a0154e332ed5af0c646c189f82678d5c15d3b1550ca0383645b: Status 404 returned error can't find the container with id f52994bb928b4a0154e332ed5af0c646c189f82678d5c15d3b1550ca0383645b
Sep 11 12:24:33.401477 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:24:33.401456    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 12:24:34.233363 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:24:34.233280    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7b8f67d746-765t6" event={"ID":"dc0ee18c-f3fc-4dca-9d6f-59b1407f6eea","Type":"ContainerStarted","Data":"5f409d63a0841337555350a1a296fd83767a1844572fdaa83152059fec9b14ee"}
Sep 11 12:24:34.233363 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:24:34.233331    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7b8f67d746-765t6" event={"ID":"dc0ee18c-f3fc-4dca-9d6f-59b1407f6eea","Type":"ContainerStarted","Data":"f52994bb928b4a0154e332ed5af0c646c189f82678d5c15d3b1550ca0383645b"}
Sep 11 12:24:34.233829 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:24:34.233431    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7b8f67d746-765t6"
Sep 11 12:24:34.254585 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:24:34.254531    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7b8f67d746-765t6" podStartSLOduration=1.7801260559999998 podStartE2EDuration="2.254513399s" totalImagesPullingTime="474.387343ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:24:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:24:34.251602157 +0000 UTC m=+2178.149043904" watchObservedRunningTime="2026-09-11 12:24:34.254513399 +0000 UTC m=+2178.151955145"
Sep 11 12:24:40.243822 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:24:40.243787    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7b8f67d746-765t6"
Sep 11 12:24:40.262694 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:24:40.262661    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:24:47.162444 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:24:47.162407    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:24:52.763228 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:24:52.763184    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:24:59.961473 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:24:59.961432    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-znhkx"]
Sep 11 12:25:06.043088 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:06.043045    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:25:08.759564 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:08.759534    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-t4h64_1d089f0b-5e09-440a-a8e5-cbe5bf59b1f2/manager/0.log"
Sep 11 12:25:08.901659 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:08.901628    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-vkf9d_946b34d0-c922-458b-8b7e-b6de54bbcd05/manager/0.log"
Sep 11 12:25:09.196367 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:09.196336    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-4zb6j_5348a878-29ad-49e6-a1cf-aab044f10e14/server/0.log"
Sep 11 12:25:09.335112 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:09.335084    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-mvbmv_59b9266e-e066-4bfc-8a4e-505d6fb6f7dd/manager/0.log"
Sep 11 12:25:09.459511 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:09.459424    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7fb8846894-drb98_b5a20575-88da-4ed8-ab26-487b7ef2ae80/manager/0.log"
Sep 11 12:25:09.466281 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:09.466255    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7fb8846894-drb98_b5a20575-88da-4ed8-ab26-487b7ef2ae80/copy-manifests/0.log"
Sep 11 12:25:10.120949 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:10.120916    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-7b8f67d746-765t6_dc0ee18c-f3fc-4dca-9d6f-59b1407f6eea/maas-api/0.log"
Sep 11 12:25:10.354537 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:10.354505    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29818800-xtcbc_94cb2699-87c3-4efa-bedf-75341e1bd10f/cleanup/0.log"
Sep 11 12:25:11.578902 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:11.578873    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-wwjmx_14b00de4-0665-4db5-90c6-5af92818fcc9/manager/0.log"
Sep 11 12:25:12.072844 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:12.072811    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-znhkx_a2f763ac-c4a1-4ca5-866a-a10c3c64a893/limitador/0.log"
Sep 11 12:25:12.197351 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:12.197318    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-stlxq_fe3781b9-2a1e-4f3f-93ea-29b2a27e85e6/manager/0.log"
Sep 11 12:25:12.681529 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:12.681495    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-zc7qw_93e10029-98cd-4ead-b26d-ed657714e363/discovery/0.log"
Sep 11 12:25:12.925039 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:12.925003    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-57dff8dc5-xn758_47d3ab09-6e98-4185-8c2c-4ee78873af56/kube-auth-proxy/0.log"
Sep 11 12:25:13.451030 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:13.450981    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7b5c4b4df6-24w55_6e8b4926-8dc4-4622-9882-8d039646c1c0/router/0.log"
Sep 11 12:25:13.973277 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:13.973238    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-gcbd8_84f2fd48-b014-4d96-b247-44a77932b73c/main/0.log"
Sep 11 12:25:14.352466 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:14.352422    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58ppm_c868518c-3ff5-46e5-8186-a244b09bd377/main/0.log"
Sep 11 12:25:17.033027 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:17.032995    2579 scope.go:117] "RemoveContainer" containerID="9f53ee31ef3b94851b23762d1e92b4c6b15a1a922f9013ccb05a1447772dbd9e"
Sep 11 12:25:17.041358 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:17.041293    2579 scope.go:117] "RemoveContainer" containerID="712b37554163be72f274a047bce54768bdbf503ced53b5536d34a17141a17061"
Sep 11 12:25:20.023193 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:20.023154    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:25:20.090196 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:20.090156    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log" limit=1024
Sep 11 12:25:23.117157 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:23.117126    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-79ltx_70994cc4-c221-4ab7-99ec-aed7c86955de/global-pull-secret-syncer/0.log"
Sep 11 12:25:23.264823 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:23.264789    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-dxf9l_43cde4a1-c7f9-443d-9f2f-0ab809b5d25a/konnectivity-agent/0.log"
Sep 11 12:25:23.388536 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:23.388416    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-145.ec2.internal_3d9525f1a3218f8a340d89c0b17b6cdb/haproxy/0.log"
Sep 11 12:25:27.628462 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:27.628421    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-wwjmx_14b00de4-0665-4db5-90c6-5af92818fcc9/manager/0.log"
Sep 11 12:25:27.757899 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:27.757870    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-znhkx_a2f763ac-c4a1-4ca5-866a-a10c3c64a893/limitador/0.log"
Sep 11 12:25:27.848705 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:27.848668    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-stlxq_fe3781b9-2a1e-4f3f-93ea-29b2a27e85e6/manager/0.log"
Sep 11 12:25:29.772085 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:29.771997    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5f945db5-479mq_cab42ad7-002b-4a76-971d-446580ae9467/metrics-server/0.log"
Sep 11 12:25:30.049551 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:30.049459    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-p97jw_0af7a15f-31e5-4f13-85f5-0879569febcc/node-exporter/0.log"
Sep 11 12:25:30.074435 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:30.074403    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-p97jw_0af7a15f-31e5-4f13-85f5-0879569febcc/kube-rbac-proxy/0.log"
Sep 11 12:25:30.099758 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:30.099738    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-p97jw_0af7a15f-31e5-4f13-85f5-0879569febcc/init-textfile/0.log"
Sep 11 12:25:30.144205 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:30.144172    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-j8pjk_a0a5db0b-9a22-4973-b161-09d4675e0843/kube-rbac-proxy-main/0.log"
Sep 11 12:25:30.182295 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:30.182264    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-j8pjk_a0a5db0b-9a22-4973-b161-09d4675e0843/kube-rbac-proxy-self/0.log"
Sep 11 12:25:30.218973 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:30.218940    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-j8pjk_a0a5db0b-9a22-4973-b161-09d4675e0843/openshift-state-metrics/0.log"
Sep 11 12:25:30.269577 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:30.269541    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9e57c56f-2584-4328-b946-02b2ab076a97/prometheus/0.log"
Sep 11 12:25:30.308064 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:30.307980    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9e57c56f-2584-4328-b946-02b2ab076a97/config-reloader/0.log"
Sep 11 12:25:30.335341 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:30.335315    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9e57c56f-2584-4328-b946-02b2ab076a97/thanos-sidecar/0.log"
Sep 11 12:25:30.363617 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:30.363593    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9e57c56f-2584-4328-b946-02b2ab076a97/kube-rbac-proxy-web/0.log"
Sep 11 12:25:30.389332 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:30.389305    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9e57c56f-2584-4328-b946-02b2ab076a97/kube-rbac-proxy/0.log"
Sep 11 12:25:30.413432 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:30.413381    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9e57c56f-2584-4328-b946-02b2ab076a97/kube-rbac-proxy-thanos/0.log"
Sep 11 12:25:30.438995 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:30.438970    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9e57c56f-2584-4328-b946-02b2ab076a97/init-config-reloader/0.log"
Sep 11 12:25:30.535676 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:30.535651    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5864cd99d7-t8t5g_4bfe732c-eb81-4d2a-b1ae-213050a8205c/prometheus-operator-admission-webhook/0.log"
Sep 11 12:25:30.671753 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:30.671724    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-56bbffc5d9-7z4hv_3da41047-d209-4ff5-afac-ad41cf4333c7/thanos-query/0.log"
Sep 11 12:25:30.697132 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:30.697097    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-56bbffc5d9-7z4hv_3da41047-d209-4ff5-afac-ad41cf4333c7/kube-rbac-proxy-web/0.log"
Sep 11 12:25:30.726290 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:30.726258    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-56bbffc5d9-7z4hv_3da41047-d209-4ff5-afac-ad41cf4333c7/kube-rbac-proxy/0.log"
Sep 11 12:25:30.747898 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:30.747860    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-56bbffc5d9-7z4hv_3da41047-d209-4ff5-afac-ad41cf4333c7/prom-label-proxy/0.log"
Sep 11 12:25:30.774078 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:30.774042    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-56bbffc5d9-7z4hv_3da41047-d209-4ff5-afac-ad41cf4333c7/kube-rbac-proxy-rules/0.log"
Sep 11 12:25:30.799756 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:30.799729    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-56bbffc5d9-7z4hv_3da41047-d209-4ff5-afac-ad41cf4333c7/kube-rbac-proxy-metrics/0.log"
Sep 11 12:25:32.246181 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:32.246149    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-5dcbc7df6b-wk5n5_7d184dcd-3ce1-469a-b8e3-9db0d7c09255/networking-console-plugin/0.log"
Sep 11 12:25:32.884892 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:32.884844    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-ch96n_1f0f9fa5-cbd1-468b-ae81-82b48c101d37/console-operator/2.log"
Sep 11 12:25:32.892043 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:32.892014    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-ch96n_1f0f9fa5-cbd1-468b-ae81-82b48c101d37/console-operator/3.log"
Sep 11 12:25:32.978862 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:32.978828    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-vdhzr/perf-node-gather-daemonset-m8dfz"]
Sep 11 12:25:32.979367 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:32.979347    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45e86e88-5ccd-4c4c-8d31-8350e844f996" containerName="istio-proxy"
Sep 11 12:25:32.979643 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:32.979370    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="45e86e88-5ccd-4c4c-8d31-8350e844f996" containerName="istio-proxy"
Sep 11 12:25:32.982819 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:32.982802    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vdhzr/perf-node-gather-daemonset-m8dfz"
Sep 11 12:25:32.985696 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:32.985673    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vdhzr\"/\"kube-root-ca.crt\""
Sep 11 12:25:32.985696 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:32.985695    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vdhzr\"/\"openshift-service-ca.crt\""
Sep 11 12:25:32.986701 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:32.986680    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-vdhzr\"/\"default-dockercfg-m29sz\""
Sep 11 12:25:32.992483 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:32.992462    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vdhzr/perf-node-gather-daemonset-m8dfz"]
Sep 11 12:25:33.107221 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:33.107191    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/45c525c6-7d44-46c3-962e-7c355f9b74d2-proc\") pod \"perf-node-gather-daemonset-m8dfz\" (UID: \"45c525c6-7d44-46c3-962e-7c355f9b74d2\") " pod="openshift-must-gather-vdhzr/perf-node-gather-daemonset-m8dfz"
Sep 11 12:25:33.107450 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:33.107234    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/45c525c6-7d44-46c3-962e-7c355f9b74d2-sys\") pod \"perf-node-gather-daemonset-m8dfz\" (UID: \"45c525c6-7d44-46c3-962e-7c355f9b74d2\") " pod="openshift-must-gather-vdhzr/perf-node-gather-daemonset-m8dfz"
Sep 11 12:25:33.107450 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:33.107268    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2sqxh\" (UniqueName: \"kubernetes.io/projected/45c525c6-7d44-46c3-962e-7c355f9b74d2-kube-api-access-2sqxh\") pod \"perf-node-gather-daemonset-m8dfz\" (UID: \"45c525c6-7d44-46c3-962e-7c355f9b74d2\") " pod="openshift-must-gather-vdhzr/perf-node-gather-daemonset-m8dfz"
Sep 11 12:25:33.107450 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:33.107351    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/45c525c6-7d44-46c3-962e-7c355f9b74d2-podres\") pod \"perf-node-gather-daemonset-m8dfz\" (UID: \"45c525c6-7d44-46c3-962e-7c355f9b74d2\") " pod="openshift-must-gather-vdhzr/perf-node-gather-daemonset-m8dfz"
Sep 11 12:25:33.107450 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:33.107376    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/45c525c6-7d44-46c3-962e-7c355f9b74d2-lib-modules\") pod \"perf-node-gather-daemonset-m8dfz\" (UID: \"45c525c6-7d44-46c3-962e-7c355f9b74d2\") " pod="openshift-must-gather-vdhzr/perf-node-gather-daemonset-m8dfz"
Sep 11 12:25:33.208241 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:33.208141    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/45c525c6-7d44-46c3-962e-7c355f9b74d2-proc\") pod \"perf-node-gather-daemonset-m8dfz\" (UID: \"45c525c6-7d44-46c3-962e-7c355f9b74d2\") " pod="openshift-must-gather-vdhzr/perf-node-gather-daemonset-m8dfz"
Sep 11 12:25:33.208241 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:33.208206    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/45c525c6-7d44-46c3-962e-7c355f9b74d2-sys\") pod \"perf-node-gather-daemonset-m8dfz\" (UID: \"45c525c6-7d44-46c3-962e-7c355f9b74d2\") " pod="openshift-must-gather-vdhzr/perf-node-gather-daemonset-m8dfz"
Sep 11 12:25:33.208511 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:33.208251    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2sqxh\" (UniqueName: \"kubernetes.io/projected/45c525c6-7d44-46c3-962e-7c355f9b74d2-kube-api-access-2sqxh\") pod \"perf-node-gather-daemonset-m8dfz\" (UID: \"45c525c6-7d44-46c3-962e-7c355f9b74d2\") " pod="openshift-must-gather-vdhzr/perf-node-gather-daemonset-m8dfz"
Sep 11 12:25:33.208511 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:33.208281    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/45c525c6-7d44-46c3-962e-7c355f9b74d2-proc\") pod \"perf-node-gather-daemonset-m8dfz\" (UID: \"45c525c6-7d44-46c3-962e-7c355f9b74d2\") " pod="openshift-must-gather-vdhzr/perf-node-gather-daemonset-m8dfz"
Sep 11 12:25:33.208511 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:33.208342    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/45c525c6-7d44-46c3-962e-7c355f9b74d2-podres\") pod \"perf-node-gather-daemonset-m8dfz\" (UID: \"45c525c6-7d44-46c3-962e-7c355f9b74d2\") " pod="openshift-must-gather-vdhzr/perf-node-gather-daemonset-m8dfz"
Sep 11 12:25:33.208511 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:33.208377    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/45c525c6-7d44-46c3-962e-7c355f9b74d2-lib-modules\") pod \"perf-node-gather-daemonset-m8dfz\" (UID: \"45c525c6-7d44-46c3-962e-7c355f9b74d2\") " pod="openshift-must-gather-vdhzr/perf-node-gather-daemonset-m8dfz"
Sep 11 12:25:33.208511 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:33.208342    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/45c525c6-7d44-46c3-962e-7c355f9b74d2-sys\") pod \"perf-node-gather-daemonset-m8dfz\" (UID: \"45c525c6-7d44-46c3-962e-7c355f9b74d2\") " pod="openshift-must-gather-vdhzr/perf-node-gather-daemonset-m8dfz"
Sep 11 12:25:33.208719 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:33.208515    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/45c525c6-7d44-46c3-962e-7c355f9b74d2-podres\") pod \"perf-node-gather-daemonset-m8dfz\" (UID: \"45c525c6-7d44-46c3-962e-7c355f9b74d2\") " pod="openshift-must-gather-vdhzr/perf-node-gather-daemonset-m8dfz"
Sep 11 12:25:33.208719 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:33.208516    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/45c525c6-7d44-46c3-962e-7c355f9b74d2-lib-modules\") pod \"perf-node-gather-daemonset-m8dfz\" (UID: \"45c525c6-7d44-46c3-962e-7c355f9b74d2\") " pod="openshift-must-gather-vdhzr/perf-node-gather-daemonset-m8dfz"
Sep 11 12:25:33.217744 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:33.217721    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2sqxh\" (UniqueName: \"kubernetes.io/projected/45c525c6-7d44-46c3-962e-7c355f9b74d2-kube-api-access-2sqxh\") pod \"perf-node-gather-daemonset-m8dfz\" (UID: \"45c525c6-7d44-46c3-962e-7c355f9b74d2\") " pod="openshift-must-gather-vdhzr/perf-node-gather-daemonset-m8dfz"
Sep 11 12:25:33.293719 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:33.293680    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vdhzr/perf-node-gather-daemonset-m8dfz"
Sep 11 12:25:33.573715 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:33.573687    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vdhzr/perf-node-gather-daemonset-m8dfz"]
Sep 11 12:25:33.576216 ip-10-0-142-145 kubenswrapper[2579]: W0911 12:25:33.576189    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod45c525c6_7d44_46c3_962e_7c355f9b74d2.slice/crio-ba2362860947b94c7bb064ed493cbceb5a6b6a481603e48182004af53f14474a WatchSource:0}: Error finding container ba2362860947b94c7bb064ed493cbceb5a6b6a481603e48182004af53f14474a: Status 404 returned error can't find the container with id ba2362860947b94c7bb064ed493cbceb5a6b6a481603e48182004af53f14474a
Sep 11 12:25:34.550174 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:34.550134    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vdhzr/perf-node-gather-daemonset-m8dfz" event={"ID":"45c525c6-7d44-46c3-962e-7c355f9b74d2","Type":"ContainerStarted","Data":"8cd0b0ad9f0eb666b3a3dc9d87d2963e6d5dde1c51451d875d531e652370ab60"}
Sep 11 12:25:34.550675 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:34.550213    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vdhzr/perf-node-gather-daemonset-m8dfz" event={"ID":"45c525c6-7d44-46c3-962e-7c355f9b74d2","Type":"ContainerStarted","Data":"ba2362860947b94c7bb064ed493cbceb5a6b6a481603e48182004af53f14474a"}
Sep 11 12:25:34.550675 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:34.550298    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-vdhzr/perf-node-gather-daemonset-m8dfz"
Sep 11 12:25:34.569145 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:34.569089    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-vdhzr/perf-node-gather-daemonset-m8dfz" podStartSLOduration=2.569075812 podStartE2EDuration="2.569075812s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 12:25:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:25:34.568657266 +0000 UTC m=+2238.466099011" watchObservedRunningTime="2026-09-11 12:25:34.569075812 +0000 UTC m=+2238.466517557"
Sep 11 12:25:35.450138 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:35.450111    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-pmhcz_9246798b-82ff-4c57-97ff-ce4315e9a904/dns/0.log"
Sep 11 12:25:35.474853 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:35.474821    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-pmhcz_9246798b-82ff-4c57-97ff-ce4315e9a904/kube-rbac-proxy/0.log"
Sep 11 12:25:35.503449 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:35.503417    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-2jqbl_cdaded97-b0ac-4688-8721-813fc905c103/dns-node-resolver/0.log"
Sep 11 12:25:36.175177 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:36.175147    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-t7cd7_0f52f2ea-a695-422d-a16c-fa3db1f93041/node-ca/0.log"
Sep 11 12:25:37.588684 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:37.588653    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-zc7qw_93e10029-98cd-4ead-b26d-ed657714e363/discovery/0.log"
Sep 11 12:25:37.635909 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:37.635881    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-57dff8dc5-xn758_47d3ab09-6e98-4185-8c2c-4ee78873af56/kube-auth-proxy/0.log"
Sep 11 12:25:37.788196 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:37.788163    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7b5c4b4df6-24w55_6e8b4926-8dc4-4622-9882-8d039646c1c0/router/0.log"
Sep 11 12:25:38.800673 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:38.800597    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qvss2_9bc72241-e4f1-41eb-84dc-994cccf593c2/serve-healthcheck-canary/0.log"
Sep 11 12:25:39.314921 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:39.314883    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-9l6kz_7f0c67d9-ee3f-4f68-9e53-2b6e8b238d5e/insights-operator/0.log"
Sep 11 12:25:39.315308 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:39.315277    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-9l6kz_7f0c67d9-ee3f-4f68-9e53-2b6e8b238d5e/insights-operator/1.log"
Sep 11 12:25:39.499138 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:39.499106    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kmg7h_4b394c7d-887e-48b3-bb4b-9dea65f2c51b/kube-rbac-proxy/0.log"
Sep 11 12:25:39.521900 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:39.521869    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kmg7h_4b394c7d-887e-48b3-bb4b-9dea65f2c51b/exporter/0.log"
Sep 11 12:25:39.548062 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:39.548035    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kmg7h_4b394c7d-887e-48b3-bb4b-9dea65f2c51b/extractor/0.log"
Sep 11 12:25:40.565622 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:40.565597    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-vdhzr/perf-node-gather-daemonset-m8dfz"
Sep 11 12:25:41.791382 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:41.791347    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-t4h64_1d089f0b-5e09-440a-a8e5-cbe5bf59b1f2/manager/0.log"
Sep 11 12:25:42.126357 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:42.126324    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-vkf9d_946b34d0-c922-458b-8b7e-b6de54bbcd05/manager/0.log"
Sep 11 12:25:42.284982 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:42.284948    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-4zb6j_5348a878-29ad-49e6-a1cf-aab044f10e14/server/0.log"
Sep 11 12:25:42.423595 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:42.423505    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-mvbmv_59b9266e-e066-4bfc-8a4e-505d6fb6f7dd/manager/0.log"
Sep 11 12:25:42.452478 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:42.452451    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7fb8846894-drb98_b5a20575-88da-4ed8-ab26-487b7ef2ae80/manager/0.log"
Sep 11 12:25:42.475458 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:42.475431    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7fb8846894-drb98_b5a20575-88da-4ed8-ab26-487b7ef2ae80/copy-manifests/0.log"
Sep 11 12:25:43.345646 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:43.345614    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-zc7qw_93e10029-98cd-4ead-b26d-ed657714e363/discovery/0.log"
Sep 11 12:25:43.377360 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:43.377329    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-57dff8dc5-xn758_47d3ab09-6e98-4185-8c2c-4ee78873af56/kube-auth-proxy/0.log"
Sep 11 12:25:43.570033 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:43.570006    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7b5c4b4df6-24w55_6e8b4926-8dc4-4622-9882-8d039646c1c0/router/0.log"
Sep 11 12:25:45.552463 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:45.552432    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-zc7qw_93e10029-98cd-4ead-b26d-ed657714e363/discovery/0.log"
Sep 11 12:25:45.584078 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:45.584049    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-57dff8dc5-xn758_47d3ab09-6e98-4185-8c2c-4ee78873af56/kube-auth-proxy/0.log"
Sep 11 12:25:45.689136 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:45.689109    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7b5c4b4df6-24w55_6e8b4926-8dc4-4622-9882-8d039646c1c0/router/0.log"
Sep 11 12:25:49.457077 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:49.457041    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-6dtzr_b6664260-08f2-4521-93a3-968b672e69da/migrator/0.log"
Sep 11 12:25:49.481541 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:49.481499    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-6dtzr_b6664260-08f2-4521-93a3-968b672e69da/graceful-termination/0.log"
Sep 11 12:25:50.969315 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:50.969284    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4jthr_7872368e-69ce-4a5c-9773-e2d02b8f06ba/kube-multus-additional-cni-plugins/0.log"
Sep 11 12:25:50.992672 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:50.992643    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4jthr_7872368e-69ce-4a5c-9773-e2d02b8f06ba/egress-router-binary-copy/0.log"
Sep 11 12:25:51.016293 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:51.016262    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4jthr_7872368e-69ce-4a5c-9773-e2d02b8f06ba/cni-plugins/0.log"
Sep 11 12:25:51.037808 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:51.037778    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4jthr_7872368e-69ce-4a5c-9773-e2d02b8f06ba/bond-cni-plugin/0.log"
Sep 11 12:25:51.061369 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:51.061335    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4jthr_7872368e-69ce-4a5c-9773-e2d02b8f06ba/routeoverride-cni/0.log"
Sep 11 12:25:51.084116 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:51.084083    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4jthr_7872368e-69ce-4a5c-9773-e2d02b8f06ba/whereabouts-cni-bincopy/0.log"
Sep 11 12:25:51.111752 ip-10-0-142-145 kubenswrapper[2579]: I0911 12:25:51.111717    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4jthr_7872368e-69ce-4a5c-9773-e2d02b8f06ba/whereabouts-cni/0.log"