Sep 09 16:08:44.477808 ip-10-0-128-81 systemd[1]: Starting Kubernetes Kubelet...
Sep 09 16:08:44.836038 ip-10-0-128-81 kubenswrapper[2567]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 09 16:08:44.836038 ip-10-0-128-81 kubenswrapper[2567]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 09 16:08:44.836038 ip-10-0-128-81 kubenswrapper[2567]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 09 16:08:44.836038 ip-10-0-128-81 kubenswrapper[2567]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 09 16:08:44.836038 ip-10-0-128-81 kubenswrapper[2567]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 09 16:08:44.837609 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.837523    2567 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 09 16:08:44.841993 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.841976    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 16:08:44.841993 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.841992    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 16:08:44.841993 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.841996    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 16:08:44.842090 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842000    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 16:08:44.842090 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842003    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 16:08:44.842090 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842006    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 16:08:44.842090 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842009    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 16:08:44.842090 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842012    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 16:08:44.842090 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842015    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 16:08:44.842090 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842018    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 16:08:44.842090 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842021    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 16:08:44.842090 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842023    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 16:08:44.842090 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842026    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 16:08:44.842090 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842028    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 16:08:44.842090 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842031    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 16:08:44.842090 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842039    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 16:08:44.842090 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842042    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 16:08:44.842090 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842044    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 16:08:44.842090 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842047    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 16:08:44.842090 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842049    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 16:08:44.842090 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842051    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 16:08:44.842090 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842054    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 16:08:44.842090 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842057    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 16:08:44.842591 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842060    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 16:08:44.842591 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842064    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 16:08:44.842591 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842068    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 16:08:44.842591 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842071    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 16:08:44.842591 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842074    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 16:08:44.842591 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842077    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 16:08:44.842591 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842079    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 16:08:44.842591 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842082    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 16:08:44.842591 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842084    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 16:08:44.842591 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842087    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 16:08:44.842591 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842090    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 16:08:44.842591 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842093    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 16:08:44.842591 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842096    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 16:08:44.842591 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842099    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 16:08:44.842591 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842102    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 16:08:44.842591 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842105    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 16:08:44.842591 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842108    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 16:08:44.842591 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842111    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 16:08:44.842591 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842113    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 16:08:44.842591 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842116    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 16:08:44.843106 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842118    2567 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 16:08:44.843106 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842121    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 16:08:44.843106 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842123    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 16:08:44.843106 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842126    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 16:08:44.843106 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842128    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 16:08:44.843106 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842131    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 16:08:44.843106 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842133    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 16:08:44.843106 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842135    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 16:08:44.843106 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842138    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 16:08:44.843106 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842141    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 16:08:44.843106 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842143    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 16:08:44.843106 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842146    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 16:08:44.843106 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842148    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 16:08:44.843106 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842152    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 16:08:44.843106 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842156    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 16:08:44.843106 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842160    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 16:08:44.843106 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842164    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 16:08:44.843106 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842167    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 16:08:44.843106 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842169    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 16:08:44.843106 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842172    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 16:08:44.843637 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842175    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 16:08:44.843637 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842177    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 16:08:44.843637 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842179    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 16:08:44.843637 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842182    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 16:08:44.843637 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842184    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 16:08:44.843637 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842187    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 16:08:44.843637 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842189    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 16:08:44.843637 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842191    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 16:08:44.843637 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842201    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 16:08:44.843637 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842203    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 16:08:44.843637 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842206    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 16:08:44.843637 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842208    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 16:08:44.843637 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842210    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 16:08:44.843637 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842213    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 16:08:44.843637 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842215    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 16:08:44.843637 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842218    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 16:08:44.843637 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842220    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 16:08:44.843637 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842223    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 16:08:44.843637 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842226    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 16:08:44.844105 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842229    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 16:08:44.844105 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842233    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 16:08:44.844105 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842236    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 16:08:44.844105 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842239    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 16:08:44.844105 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842242    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 16:08:44.844105 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842245    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 16:08:44.844105 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842248    2567 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 16:08:44.844105 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842251    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 16:08:44.844105 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842254    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 16:08:44.844105 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842256    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 16:08:44.844105 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842259    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 16:08:44.844105 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842262    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 16:08:44.844105 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842264    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 16:08:44.844105 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842266    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 16:08:44.844105 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842269    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 16:08:44.844105 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842271    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 16:08:44.844105 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.842274    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 16:08:44.844105 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843011    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 16:08:44.844105 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843017    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 16:08:44.844594 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843019    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 16:08:44.844594 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843022    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 16:08:44.844594 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843025    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 16:08:44.844594 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843033    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 16:08:44.844594 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843036    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 16:08:44.844594 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843038    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 16:08:44.844594 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843041    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 16:08:44.844594 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843044    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 16:08:44.844594 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843046    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 16:08:44.844594 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843049    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 16:08:44.844594 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843052    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 16:08:44.844594 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843055    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 16:08:44.844594 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843057    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 16:08:44.844594 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843060    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 16:08:44.844594 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843062    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 16:08:44.844594 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843065    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 16:08:44.844594 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843068    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 16:08:44.844594 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843070    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 16:08:44.844594 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843073    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 16:08:44.845097 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843078    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 16:08:44.845097 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843082    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 16:08:44.845097 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843086    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 16:08:44.845097 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843089    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 16:08:44.845097 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843092    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 16:08:44.845097 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843095    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 16:08:44.845097 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843098    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 16:08:44.845097 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843101    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 16:08:44.845097 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843103    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 16:08:44.845097 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843106    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 16:08:44.845097 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843109    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 16:08:44.845097 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843112    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 16:08:44.845097 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843114    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 16:08:44.845097 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843117    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 16:08:44.845097 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843119    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 16:08:44.845097 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843122    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 16:08:44.845097 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843124    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 16:08:44.845097 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843133    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 16:08:44.845097 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843135    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 16:08:44.845615 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843137    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 16:08:44.845615 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843140    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 16:08:44.845615 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843143    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 16:08:44.845615 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843145    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 16:08:44.845615 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843148    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 16:08:44.845615 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843151    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 16:08:44.845615 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843153    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 16:08:44.845615 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843156    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 16:08:44.845615 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843158    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 16:08:44.845615 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843161    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 16:08:44.845615 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843163    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 16:08:44.845615 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843166    2567 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 16:08:44.845615 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843168    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 16:08:44.845615 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843171    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 16:08:44.845615 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843174    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 16:08:44.845615 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843176    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 16:08:44.845615 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843179    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 16:08:44.845615 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843181    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 16:08:44.845615 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843184    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 16:08:44.845615 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843186    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 16:08:44.846112 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843189    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 16:08:44.846112 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843191    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 16:08:44.846112 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843194    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 16:08:44.846112 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843196    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 16:08:44.846112 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843199    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 16:08:44.846112 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843201    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 16:08:44.846112 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843204    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 16:08:44.846112 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843207    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 16:08:44.846112 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843211    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 16:08:44.846112 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843214    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 16:08:44.846112 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843216    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 16:08:44.846112 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843224    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 16:08:44.846112 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843227    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 16:08:44.846112 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843229    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 16:08:44.846112 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843232    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 16:08:44.846112 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843235    2567 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 16:08:44.846112 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843238    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 16:08:44.846112 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843242    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 16:08:44.846112 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843244    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 16:08:44.846112 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843247    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 16:08:44.846609 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843249    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 16:08:44.846609 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843252    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 16:08:44.846609 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843255    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 16:08:44.846609 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843257    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 16:08:44.846609 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843260    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 16:08:44.846609 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843262    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 16:08:44.846609 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843265    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 16:08:44.846609 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843267    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 16:08:44.846609 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843270    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 16:08:44.846609 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843273    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 16:08:44.846609 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843275    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 16:08:44.846609 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843279    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 16:08:44.846609 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843282    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 16:08:44.846609 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843285    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 16:08:44.846609 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843288    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 16:08:44.846609 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843291    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 16:08:44.846609 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843294    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 16:08:44.846609 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843297    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 16:08:44.846609 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843299    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 16:08:44.847080 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843775    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 16:08:44.847080 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843782    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 16:08:44.847080 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843785    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 16:08:44.847080 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843790    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 16:08:44.847080 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843801    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 16:08:44.847080 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843803    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 16:08:44.847080 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843806    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 16:08:44.847080 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843808    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 16:08:44.847080 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843811    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 16:08:44.847080 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843814    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 16:08:44.847080 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843817    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 16:08:44.847080 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843820    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 16:08:44.847080 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843822    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 16:08:44.847080 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843825    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 16:08:44.847080 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843828    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 16:08:44.847080 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843831    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 16:08:44.847080 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843834    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 16:08:44.847080 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843836    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 16:08:44.847080 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843839    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 16:08:44.847563 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843841    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 16:08:44.847563 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843844    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 16:08:44.847563 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843846    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 16:08:44.847563 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843849    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 16:08:44.847563 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843851    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 16:08:44.847563 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843854    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 16:08:44.847563 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843857    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 16:08:44.847563 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843859    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 16:08:44.847563 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843861    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 16:08:44.847563 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843864    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 16:08:44.847563 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843866    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 16:08:44.847563 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843869    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 16:08:44.847563 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843871    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 16:08:44.847563 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843874    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 16:08:44.847563 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843876    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 16:08:44.847563 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843879    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 16:08:44.847563 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843881    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 16:08:44.847563 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843884    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 16:08:44.847563 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843892    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 16:08:44.847563 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843895    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 16:08:44.848040 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843897    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 16:08:44.848040 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843900    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 16:08:44.848040 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843902    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 16:08:44.848040 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843905    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 16:08:44.848040 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843908    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 16:08:44.848040 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843910    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 16:08:44.848040 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843912    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 16:08:44.848040 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843917    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 16:08:44.848040 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843921    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 16:08:44.848040 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843924    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 16:08:44.848040 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843927    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 16:08:44.848040 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843930    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 16:08:44.848040 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843932    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 16:08:44.848040 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843935    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 16:08:44.848040 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843937    2567 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 16:08:44.848040 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843940    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 16:08:44.848040 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843942    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 16:08:44.848040 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843944    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 16:08:44.848040 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843947    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 16:08:44.848040 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843949    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 16:08:44.848546 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843952    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 16:08:44.848546 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843955    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 16:08:44.848546 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843957    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 16:08:44.848546 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843961    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 16:08:44.848546 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843965    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 16:08:44.848546 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843968    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 16:08:44.848546 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843971    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 16:08:44.848546 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843973    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 16:08:44.848546 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843976    2567 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 16:08:44.848546 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843979    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 16:08:44.848546 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843981    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 16:08:44.848546 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843989    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 16:08:44.848546 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843992    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 16:08:44.848546 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843994    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 16:08:44.848546 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.843997    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 16:08:44.848546 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.844000    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 16:08:44.848546 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.844002    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 16:08:44.848546 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.844004    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 16:08:44.848546 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.844007    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 16:08:44.848546 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.844010    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 16:08:44.849029 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.844013    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 16:08:44.849029 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.844016    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 16:08:44.849029 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.844018    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 16:08:44.849029 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.844021    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 16:08:44.849029 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.844023    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 16:08:44.849029 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.844026    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 16:08:44.849029 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.844028    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 16:08:44.849029 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.844031    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 16:08:44.849029 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.844033    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 16:08:44.849029 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.844036    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 16:08:44.849029 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.844039    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 16:08:44.849029 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.844041    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 16:08:44.849029 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.844044    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 16:08:44.849029 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.844047    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 16:08:44.849029 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.844049    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 16:08:44.849029 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.844051    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 16:08:44.849029 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.844055    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 16:08:44.849029 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.844058    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 16:08:44.849029 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.844061    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 16:08:44.849526 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.844063    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 16:08:44.849526 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844143    2567 flags.go:64] FLAG: --address="0.0.0.0"
Sep 09 16:08:44.849526 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844175    2567 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 09 16:08:44.849526 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844185    2567 flags.go:64] FLAG: --anonymous-auth="true"
Sep 09 16:08:44.849526 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844190    2567 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 09 16:08:44.849526 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844199    2567 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 09 16:08:44.849526 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844203    2567 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 09 16:08:44.849526 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844207    2567 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 09 16:08:44.849526 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844211    2567 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 09 16:08:44.849526 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844214    2567 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 09 16:08:44.849526 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844217    2567 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 09 16:08:44.849526 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844220    2567 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 09 16:08:44.849526 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844227    2567 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 09 16:08:44.849526 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844230    2567 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 09 16:08:44.849526 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844233    2567 flags.go:64] FLAG: --cgroup-root=""
Sep 09 16:08:44.849526 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844237    2567 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 09 16:08:44.849526 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844240    2567 flags.go:64] FLAG: --client-ca-file=""
Sep 09 16:08:44.849526 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844242    2567 flags.go:64] FLAG: --cloud-provider="external"
Sep 09 16:08:44.849526 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844245    2567 flags.go:64] FLAG: --cluster-dns="[]"
Sep 09 16:08:44.849526 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844251    2567 flags.go:64] FLAG: --cluster-domain=""
Sep 09 16:08:44.849526 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844254    2567 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 09 16:08:44.849526 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844257    2567 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 09 16:08:44.849526 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844260    2567 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 09 16:08:44.849526 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844263    2567 flags.go:64] FLAG: --container-log-max-files="5"
Sep 09 16:08:44.850108 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844267    2567 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 09 16:08:44.850108 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844269    2567 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 09 16:08:44.850108 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844272    2567 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 09 16:08:44.850108 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844275    2567 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 09 16:08:44.850108 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844278    2567 flags.go:64] FLAG: --contention-profiling="false"
Sep 09 16:08:44.850108 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844281    2567 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 09 16:08:44.850108 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844284    2567 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 09 16:08:44.850108 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844287    2567 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 09 16:08:44.850108 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844290    2567 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 09 16:08:44.850108 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844294    2567 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 09 16:08:44.850108 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844297    2567 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 09 16:08:44.850108 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844299    2567 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 09 16:08:44.850108 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844302    2567 flags.go:64] FLAG: --enable-load-reader="false"
Sep 09 16:08:44.850108 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844305    2567 flags.go:64] FLAG: --enable-server="true"
Sep 09 16:08:44.850108 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844313    2567 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 09 16:08:44.850108 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844319    2567 flags.go:64] FLAG: --event-burst="100"
Sep 09 16:08:44.850108 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844322    2567 flags.go:64] FLAG: --event-qps="50"
Sep 09 16:08:44.850108 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844325    2567 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 09 16:08:44.850108 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844328    2567 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 09 16:08:44.850108 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844331    2567 flags.go:64] FLAG: --eviction-hard=""
Sep 09 16:08:44.850108 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844336    2567 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 09 16:08:44.850108 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844339    2567 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 09 16:08:44.850108 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844342    2567 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 09 16:08:44.850108 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844345    2567 flags.go:64] FLAG: --eviction-soft=""
Sep 09 16:08:44.850108 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844348    2567 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 09 16:08:44.850715 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844350    2567 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 09 16:08:44.850715 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844353    2567 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 09 16:08:44.850715 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844356    2567 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 09 16:08:44.850715 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844359    2567 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 09 16:08:44.850715 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844362    2567 flags.go:64] FLAG: --fail-swap-on="true"
Sep 09 16:08:44.850715 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844365    2567 flags.go:64] FLAG: --feature-gates=""
Sep 09 16:08:44.850715 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844369    2567 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 09 16:08:44.850715 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844371    2567 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 09 16:08:44.850715 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844374    2567 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 09 16:08:44.850715 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844377    2567 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 09 16:08:44.850715 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844380    2567 flags.go:64] FLAG: --healthz-port="10248"
Sep 09 16:08:44.850715 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844383    2567 flags.go:64] FLAG: --help="false"
Sep 09 16:08:44.850715 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844386    2567 flags.go:64] FLAG: --hostname-override="ip-10-0-128-81.ec2.internal"
Sep 09 16:08:44.850715 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844389    2567 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 09 16:08:44.850715 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844392    2567 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 09 16:08:44.850715 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844394    2567 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 09 16:08:44.850715 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844398    2567 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 09 16:08:44.850715 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844401    2567 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 09 16:08:44.850715 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844404    2567 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 09 16:08:44.850715 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844407    2567 flags.go:64] FLAG: --image-service-endpoint=""
Sep 09 16:08:44.850715 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844409    2567 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 09 16:08:44.850715 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844412    2567 flags.go:64] FLAG: --kube-api-burst="100"
Sep 09 16:08:44.850715 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844421    2567 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 09 16:08:44.851338 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844424    2567 flags.go:64] FLAG: --kube-api-qps="50"
Sep 09 16:08:44.851338 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844427    2567 flags.go:64] FLAG: --kube-reserved=""
Sep 09 16:08:44.851338 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844430    2567 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 09 16:08:44.851338 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844432    2567 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 09 16:08:44.851338 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844437    2567 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 09 16:08:44.851338 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844440    2567 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 09 16:08:44.851338 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844443    2567 flags.go:64] FLAG: --lock-file=""
Sep 09 16:08:44.851338 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844445    2567 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 09 16:08:44.851338 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844448    2567 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 09 16:08:44.851338 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844464    2567 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 09 16:08:44.851338 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844470    2567 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 09 16:08:44.851338 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844473    2567 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 09 16:08:44.851338 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844475    2567 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 09 16:08:44.851338 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844478    2567 flags.go:64] FLAG: --logging-format="text"
Sep 09 16:08:44.851338 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844481    2567 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 09 16:08:44.851338 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844484    2567 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 09 16:08:44.851338 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844487    2567 flags.go:64] FLAG: --manifest-url=""
Sep 09 16:08:44.851338 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844490    2567 flags.go:64] FLAG: --manifest-url-header=""
Sep 09 16:08:44.851338 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844494    2567 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 09 16:08:44.851338 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844497    2567 flags.go:64] FLAG: --max-open-files="1000000"
Sep 09 16:08:44.851338 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844501    2567 flags.go:64] FLAG: --max-pods="110"
Sep 09 16:08:44.851338 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844504    2567 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 09 16:08:44.851338 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844507    2567 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 09 16:08:44.851338 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844509    2567 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 09 16:08:44.851338 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844512    2567 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 09 16:08:44.852009 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844515    2567 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 09 16:08:44.852009 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844518    2567 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 09 16:08:44.852009 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844521    2567 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 09 16:08:44.852009 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844528    2567 flags.go:64] FLAG: --node-status-max-images="50"
Sep 09 16:08:44.852009 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844530    2567 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 09 16:08:44.852009 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844539    2567 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 09 16:08:44.852009 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844544    2567 flags.go:64] FLAG: --pod-cidr=""
Sep 09 16:08:44.852009 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844553    2567 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 09 16:08:44.852009 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844558    2567 flags.go:64] FLAG: --pod-manifest-path=""
Sep 09 16:08:44.852009 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844561    2567 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 09 16:08:44.852009 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844564    2567 flags.go:64] FLAG: --pods-per-core="0"
Sep 09 16:08:44.852009 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844567    2567 flags.go:64] FLAG: --port="10250"
Sep 09 16:08:44.852009 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844571    2567 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 09 16:08:44.852009 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844574    2567 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0d952f27b05fe85ff"
Sep 09 16:08:44.852009 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844577    2567 flags.go:64] FLAG: --qos-reserved=""
Sep 09 16:08:44.852009 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844580    2567 flags.go:64] FLAG: --read-only-port="10255"
Sep 09 16:08:44.852009 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844583    2567 flags.go:64] FLAG: --register-node="true"
Sep 09 16:08:44.852009 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844586    2567 flags.go:64] FLAG: --register-with-taints=""
Sep 09 16:08:44.852009 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844589    2567 flags.go:64] FLAG: --registry-burst="10"
Sep 09 16:08:44.852009 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844592    2567 flags.go:64] FLAG: --registry-qps="5"
Sep 09 16:08:44.852009 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844595    2567 flags.go:64] FLAG: --reserved-cpus=""
Sep 09 16:08:44.852009 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844598    2567 flags.go:64] FLAG: --reserved-memory=""
Sep 09 16:08:44.852009 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844601    2567 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 09 16:08:44.852009 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844604    2567 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 09 16:08:44.852009 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844607    2567 flags.go:64] FLAG: --rotate-certificates="false"
Sep 09 16:08:44.852631 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844610    2567 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 09 16:08:44.852631 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844613    2567 flags.go:64] FLAG: --runonce="false"
Sep 09 16:08:44.852631 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844616    2567 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 09 16:08:44.852631 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844619    2567 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 09 16:08:44.852631 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844622    2567 flags.go:64] FLAG: --seccomp-default="false"
Sep 09 16:08:44.852631 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844625    2567 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 09 16:08:44.852631 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844628    2567 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 09 16:08:44.852631 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844630    2567 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 09 16:08:44.852631 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844633    2567 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 09 16:08:44.852631 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844636    2567 flags.go:64] FLAG: --storage-driver-password="root"
Sep 09 16:08:44.852631 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844639    2567 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 09 16:08:44.852631 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844642    2567 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 09 16:08:44.852631 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844645    2567 flags.go:64] FLAG: --storage-driver-user="root"
Sep 09 16:08:44.852631 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844649    2567 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 09 16:08:44.852631 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844652    2567 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 09 16:08:44.852631 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844661    2567 flags.go:64] FLAG: --system-cgroups=""
Sep 09 16:08:44.852631 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844663    2567 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 09 16:08:44.852631 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844669    2567 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 09 16:08:44.852631 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844672    2567 flags.go:64] FLAG: --tls-cert-file=""
Sep 09 16:08:44.852631 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844675    2567 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 09 16:08:44.852631 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844681    2567 flags.go:64] FLAG: --tls-min-version=""
Sep 09 16:08:44.852631 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844684    2567 flags.go:64] FLAG: --tls-private-key-file=""
Sep 09 16:08:44.852631 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844687    2567 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 09 16:08:44.852631 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844703    2567 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 09 16:08:44.852631 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844707    2567 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 09 16:08:44.853251 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844710    2567 flags.go:64] FLAG: --v="2"
Sep 09 16:08:44.853251 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844715    2567 flags.go:64] FLAG: --version="false"
Sep 09 16:08:44.853251 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844719    2567 flags.go:64] FLAG: --vmodule=""
Sep 09 16:08:44.853251 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844722    2567 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 09 16:08:44.853251 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.844726    2567 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 09 16:08:44.853251 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.844877    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 16:08:44.853251 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.844881    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 16:08:44.853251 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.844884    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 16:08:44.853251 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.844887    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 16:08:44.853251 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.844890    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 16:08:44.853251 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.844892    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 16:08:44.853251 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.844895    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 16:08:44.853251 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.844898    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 16:08:44.853251 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.844900    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 16:08:44.853251 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.844915    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 16:08:44.853251 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.844919    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 16:08:44.853251 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.844922    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 16:08:44.853251 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.844926    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 16:08:44.853251 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.844929    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 16:08:44.853251 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.844932    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 16:08:44.853251 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.844936    2567 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 16:08:44.853888 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.844939    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 16:08:44.853888 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.844942    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 16:08:44.853888 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.844951    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 16:08:44.853888 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.844955    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 16:08:44.853888 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.844958    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 16:08:44.853888 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.844962    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 16:08:44.853888 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.844965    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 16:08:44.853888 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.844967    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 16:08:44.853888 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.844970    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 16:08:44.853888 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.844973    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 16:08:44.853888 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.844976    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 16:08:44.853888 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.844978    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 16:08:44.853888 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.844981    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 16:08:44.853888 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.844983    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 16:08:44.853888 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.844986    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 16:08:44.853888 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.844988    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 16:08:44.853888 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.844991    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 16:08:44.853888 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.844994    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 16:08:44.853888 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.844996    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 16:08:44.854358 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.844998    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 16:08:44.854358 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.845001    2567 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 16:08:44.854358 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.845004    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 16:08:44.854358 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.845006    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 16:08:44.854358 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.845009    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 16:08:44.854358 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.845011    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 16:08:44.854358 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.845013    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 16:08:44.854358 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.845016    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 16:08:44.854358 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.845018    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 16:08:44.854358 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.845021    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 16:08:44.854358 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.845023    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 16:08:44.854358 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.845026    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 16:08:44.854358 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.845029    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 16:08:44.854358 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.845032    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 16:08:44.854358 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.845035    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 16:08:44.854358 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.845037    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 16:08:44.854358 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.845045    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 16:08:44.854358 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.845047    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 16:08:44.854358 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.845050    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 16:08:44.854358 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.845053    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 16:08:44.855116 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.845055    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 16:08:44.855116 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.845058    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 16:08:44.855116 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.845060    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 16:08:44.855116 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.845062    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 16:08:44.855116 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.845065    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 16:08:44.855116 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.845067    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 16:08:44.855116 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.845070    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 16:08:44.855116 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.845072    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 16:08:44.855116 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.845075    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 16:08:44.855116 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.845077    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 16:08:44.855116 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.845080    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 16:08:44.855116 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.845082    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 16:08:44.855116 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.845084    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 16:08:44.855116 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.845087    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 16:08:44.855116 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.845090    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 16:08:44.855116 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.845092    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 16:08:44.855116 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.845095    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 16:08:44.855116 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.845097    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 16:08:44.855116 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.845099    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 16:08:44.855116 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.845102    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 16:08:44.855843 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.845105    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 16:08:44.855843 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.845107    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 16:08:44.855843 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.845110    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 16:08:44.855843 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.845112    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 16:08:44.855843 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.845116    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 16:08:44.855843 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.845118    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 16:08:44.855843 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.845121    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 16:08:44.855843 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.845124    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 16:08:44.855843 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.845126    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 16:08:44.855843 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.845134    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 16:08:44.855843 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.845137    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 16:08:44.855843 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.845139    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 16:08:44.855843 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.845142    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 16:08:44.855843 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.845144    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 16:08:44.855843 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.845146    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 16:08:44.855843 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.845149    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 16:08:44.855843 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.845151    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 16:08:44.855843 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.845153    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 16:08:44.855843 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.845156    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 16:08:44.855843 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.845158    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 16:08:44.856350 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.845161    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 16:08:44.856350 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.845163    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 16:08:44.856350 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.845166    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 16:08:44.856350 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.845168    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 16:08:44.856350 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.845718    2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 09 16:08:44.856350 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.853254    2567 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 09 16:08:44.856350 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.853270    2567 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 09 16:08:44.856350 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.854021    2567 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 09 16:08:44.856350 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.854029    2567 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 09 16:08:44.856350 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854090    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 16:08:44.856350 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854095    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 16:08:44.856350 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854098    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 16:08:44.856350 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854101    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 16:08:44.856350 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854104    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 16:08:44.856836 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854107    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 16:08:44.856836 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854110    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 16:08:44.856836 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854113    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 16:08:44.856836 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854116    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 16:08:44.856836 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854118    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 16:08:44.856836 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854121    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 16:08:44.856836 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854123    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 16:08:44.856836 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854126    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 16:08:44.856836 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854128    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 16:08:44.856836 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854131    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 16:08:44.856836 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854134    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 16:08:44.856836 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854136    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 16:08:44.856836 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854139    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 16:08:44.856836 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854141    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 16:08:44.856836 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854143    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 16:08:44.856836 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854146    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 16:08:44.856836 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854149    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 16:08:44.856836 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854151    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 16:08:44.856836 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854154    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 16:08:44.856836 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854156    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 16:08:44.857331 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854161    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 16:08:44.857331 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854165    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 16:08:44.857331 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854168    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 16:08:44.857331 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854172    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 16:08:44.857331 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854182    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 16:08:44.857331 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854185    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 16:08:44.857331 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854188    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 16:08:44.857331 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854191    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 16:08:44.857331 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854194    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 16:08:44.857331 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854196    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 16:08:44.857331 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854199    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 16:08:44.857331 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854201    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 16:08:44.857331 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854204    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 16:08:44.857331 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854206    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 16:08:44.857331 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854209    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 16:08:44.857331 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854211    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 16:08:44.857331 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854214    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 16:08:44.857331 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854217    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 16:08:44.857331 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854219    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 16:08:44.857840 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854222    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 16:08:44.857840 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854225    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 16:08:44.857840 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854228    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 16:08:44.857840 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854231    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 16:08:44.857840 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854233    2567 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 16:08:44.857840 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854236    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 16:08:44.857840 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854239    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 16:08:44.857840 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854241    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 16:08:44.857840 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854244    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 16:08:44.857840 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854246    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 16:08:44.857840 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854248    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 16:08:44.857840 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854251    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 16:08:44.857840 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854255    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 16:08:44.857840 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854257    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 16:08:44.857840 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854260    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 16:08:44.857840 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854263    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 16:08:44.857840 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854265    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 16:08:44.857840 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854275    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 16:08:44.857840 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854283    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 16:08:44.857840 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854286    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 16:08:44.858322 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854288    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 16:08:44.858322 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854291    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 16:08:44.858322 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854293    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 16:08:44.858322 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854296    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 16:08:44.858322 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854298    2567 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 16:08:44.858322 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854300    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 16:08:44.858322 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854303    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 16:08:44.858322 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854305    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 16:08:44.858322 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854308    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 16:08:44.858322 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854310    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 16:08:44.858322 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854312    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 16:08:44.858322 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854315    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 16:08:44.858322 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854317    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 16:08:44.858322 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854320    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 16:08:44.858322 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854322    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 16:08:44.858322 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854325    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 16:08:44.858322 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854327    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 16:08:44.858322 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854330    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 16:08:44.858322 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854333    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 16:08:44.858322 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854335    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 16:08:44.858863 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854337    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 16:08:44.858863 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854340    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 16:08:44.858863 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854342    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 16:08:44.858863 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854345    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 16:08:44.858863 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854348    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 16:08:44.858863 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854350    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 16:08:44.858863 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854353    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 16:08:44.858863 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854356    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 16:08:44.858863 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854358    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 16:08:44.858863 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854361    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 16:08:44.858863 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854364    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 16:08:44.858863 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854372    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 16:08:44.858863 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854376    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 16:08:44.858863 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854380    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 16:08:44.858863 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.854383    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 16:08:44.859228 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.854388    2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 09 16:08:44.859228 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856601    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 16:08:44.859228 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856618    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 16:08:44.859228 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856623    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 16:08:44.859228 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856626    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 16:08:44.859228 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856631    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 16:08:44.859228 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856636    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 16:08:44.859228 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856639    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 16:08:44.859228 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856642    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 16:08:44.859228 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856645    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 16:08:44.859228 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856647    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 16:08:44.859228 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856650    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 16:08:44.859228 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856653    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 16:08:44.859228 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856655    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 16:08:44.859681 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856658    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 16:08:44.859681 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856661    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 16:08:44.859681 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856663    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 16:08:44.859681 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856666    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 16:08:44.859681 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856668    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 16:08:44.859681 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856671    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 16:08:44.859681 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856673    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 16:08:44.859681 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856676    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 16:08:44.859681 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856679    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 16:08:44.859681 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856681    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 16:08:44.859681 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856685    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 16:08:44.859681 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856689    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 16:08:44.859681 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856691    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 16:08:44.859681 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856694    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 16:08:44.859681 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856698    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 16:08:44.859681 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856701    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 16:08:44.859681 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856704    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 16:08:44.859681 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856706    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 16:08:44.859681 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856709    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 16:08:44.860127 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856711    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 16:08:44.860127 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856715    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 16:08:44.860127 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856718    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 16:08:44.860127 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856721    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 16:08:44.860127 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856723    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 16:08:44.860127 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856726    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 16:08:44.860127 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856728    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 16:08:44.860127 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856731    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 16:08:44.860127 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856733    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 16:08:44.860127 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856736    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 16:08:44.860127 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856738    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 16:08:44.860127 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856741    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 16:08:44.860127 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856743    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 16:08:44.860127 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856746    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 16:08:44.860127 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856748    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 16:08:44.860127 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856751    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 16:08:44.860127 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856756    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 16:08:44.860127 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856760    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 16:08:44.860127 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856765    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 16:08:44.860127 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856770    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 16:08:44.860638 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856772    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 16:08:44.860638 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856776    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 16:08:44.860638 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856779    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 16:08:44.860638 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856781    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 16:08:44.860638 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856786    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 16:08:44.860638 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856790    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 16:08:44.860638 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856793    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 16:08:44.860638 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856797    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 16:08:44.860638 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856800    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 16:08:44.860638 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856804    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 16:08:44.860638 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856809    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 16:08:44.860638 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856813    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 16:08:44.860638 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856816    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 16:08:44.860638 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856819    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 16:08:44.860638 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856822    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 16:08:44.860638 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856825    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 16:08:44.860638 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856828    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 16:08:44.860638 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856830    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 16:08:44.860638 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856832    2567 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 16:08:44.860638 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856835    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 16:08:44.861130 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856838    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 16:08:44.861130 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856841    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 16:08:44.861130 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856843    2567 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 16:08:44.861130 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856846    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 16:08:44.861130 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856849    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 16:08:44.861130 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856851    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 16:08:44.861130 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856854    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 16:08:44.861130 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856857    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 16:08:44.861130 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856859    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 16:08:44.861130 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856862    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 16:08:44.861130 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856864    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 16:08:44.861130 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856868    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 16:08:44.861130 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856871    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 16:08:44.861130 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856873    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 16:08:44.861130 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856876    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 16:08:44.861130 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856879    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 16:08:44.861130 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856882    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 16:08:44.861130 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856885    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 16:08:44.861130 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856887    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 16:08:44.861130 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856890    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 16:08:44.861632 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856892    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 16:08:44.861632 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856895    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 16:08:44.861632 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856898    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 16:08:44.861632 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856900    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 16:08:44.861632 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856903    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 16:08:44.861632 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856906    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 16:08:44.861632 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:44.856909    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 16:08:44.861632 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.856914    2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 09 16:08:44.861632 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.857047    2567 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 09 16:08:44.861632 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.859358    2567 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 09 16:08:44.861632 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.860260    2567 server.go:1019] "Starting client certificate rotation"
Sep 09 16:08:44.861632 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.860366    2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 09 16:08:44.861632 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.860403    2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 09 16:08:44.862889 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.862876    2567 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 09 16:08:44.864057 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.864041    2567 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 09 16:08:44.875549 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.875529    2567 log.go:25] "Validated CRI v1 runtime API"
Sep 09 16:08:44.878702 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.878687    2567 log.go:25] "Validated CRI v1 image API"
Sep 09 16:08:44.879624 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.879607    2567 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 09 16:08:44.884125 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.884107    2567 fs.go:135] Filesystem UUIDs: map[524896d7-4a6a-4b57-b9fe-fac3954204cf:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 7a6c7ccf-7e34-44c4-8931-425c10dbac65:/dev/nvme0n1p3]
Sep 09 16:08:44.884125 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.884124    2567 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 09 16:08:44.890990 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.890878    2567 manager.go:216] Machine: {Timestamp:2026-09-09 16:08:44.889689462 +0000 UTC m=+0.307539452 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3104471 MemoryCapacity:32812163072 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec290c66ff44135878a8688291048ad8 SystemUUID:ec290c66-ff44-1358-78a8-688291048ad8 BootID:d8a604c5-9f86-4357-9656-10680782ee78 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406081536 Type:vfs Inodes:4005391 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406081536 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:c3:85:0c:9f:4d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:c3:85:0c:9f:4d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:02:ef:52:c1:4f:be Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812163072 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 09 16:08:44.890990 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.890981    2567 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 09 16:08:44.891145 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.891076    2567 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 09 16:08:44.893852 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.893829    2567 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 09 16:08:44.894036 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.893856    2567 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-81.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 09 16:08:44.894647 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.894634    2567 topology_manager.go:138] "Creating topology manager with none policy"
Sep 09 16:08:44.894712 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.894653    2567 container_manager_linux.go:309] "Creating device plugin manager"
Sep 09 16:08:44.894712 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.894663    2567 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 09 16:08:44.895377 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.895365    2567 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 09 16:08:44.897002 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.896990    2567 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 09 16:08:44.899227 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.899210    2567 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 09 16:08:44.901053 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.901039    2567 state_mem.go:36] "Initialized new in-memory state store"
Sep 09 16:08:44.901567 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.901519    2567 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 09 16:08:44.901673 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.901630    2567 kubelet.go:486] "Attempting to sync node with API server"
Sep 09 16:08:44.901673 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.901663    2567 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 09 16:08:44.901780 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.901682    2567 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 09 16:08:44.901780 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.901698    2567 kubelet.go:398] "Adding apiserver pod source"
Sep 09 16:08:44.901780 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.901722    2567 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 09 16:08:44.902368 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.902352    2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 09 16:08:44.902445 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.902382    2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 09 16:08:44.908699 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.908677    2567 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 09 16:08:44.911376 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.911362    2567 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 09 16:08:44.911477 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.911393    2567 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 09 16:08:44.911591 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.911580    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 09 16:08:44.911628 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.911597    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 09 16:08:44.911628 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.911603    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 09 16:08:44.911628 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.911608    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 09 16:08:44.911628 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.911616    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 09 16:08:44.911628 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.911623    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 09 16:08:44.912098 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.912086    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 09 16:08:44.912098 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.912096    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 09 16:08:44.912232 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.912104    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 09 16:08:44.912232 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.912111    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 09 16:08:44.912232 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:44.912095    2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-81.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 09 16:08:44.912232 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.912120    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 09 16:08:44.912232 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.912133    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 09 16:08:44.912232 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:44.912158    2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 09 16:08:44.912818 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.912808    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 09 16:08:44.912818 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.912818    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 09 16:08:44.916721 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.916706    2567 server.go:1268] "Started kubelet"
Sep 09 16:08:44.916829 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.916801    2567 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 09 16:08:44.916887 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.916849    2567 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 09 16:08:44.916936 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.916809    2567 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 09 16:08:44.917634 ip-10-0-128-81 systemd[1]: Started Kubernetes Kubelet.
Sep 09 16:08:44.917884 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.917862    2567 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-81.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 09 16:08:44.918064 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.918045    2567 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 09 16:08:44.918940 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.918921    2567 server.go:310] "Adding debug handlers to kubelet server"
Sep 09 16:08:44.921833 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:44.921810    2567 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 09 16:08:44.923565 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.923549    2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 09 16:08:44.924603 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.924438    2567 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 09 16:08:44.924932 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:44.924856    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-81.ec2.internal\" not found"
Sep 09 16:08:44.925406 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.925317    2567 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 09 16:08:44.925406 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.925335    2567 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 09 16:08:44.925406 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.925353    2567 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 09 16:08:44.925615 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.925531    2567 reconstruct.go:97] "Volume reconstruction finished"
Sep 09 16:08:44.925615 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.925539    2567 reconciler.go:29] "Reconciler: start to sync state"
Sep 09 16:08:44.925615 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:44.924697    2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-81.ec2.internal.18d3b2d9a96f8e15  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-81.ec2.internal,UID:ip-10-0-128-81.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-81.ec2.internal,},FirstTimestamp:2026-09-09 16:08:44.916526613 +0000 UTC m=+0.334376604,LastTimestamp:2026-09-09 16:08:44.916526613 +0000 UTC m=+0.334376604,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-81.ec2.internal,}"
Sep 09 16:08:44.925910 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.925893    2567 factory.go:55] Registering systemd factory
Sep 09 16:08:44.925987 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.925913    2567 factory.go:223] Registration of the systemd container factory successfully
Sep 09 16:08:44.926317 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.926275    2567 factory.go:153] Registering CRI-O factory
Sep 09 16:08:44.926380 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.926320    2567 factory.go:223] Registration of the crio container factory successfully
Sep 09 16:08:44.926481 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.926444    2567 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 09 16:08:44.926516 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.926496    2567 factory.go:103] Registering Raw factory
Sep 09 16:08:44.926516 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.926510    2567 manager.go:1196] Started watching for new ooms in manager
Sep 09 16:08:44.927088 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.927071    2567 manager.go:349] Starting recovery of all containers
Sep 09 16:08:44.932190 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:44.932160    2567 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-81.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 09 16:08:44.932366 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:44.932342    2567 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 09 16:08:44.940253 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.940229    2567 manager.go:354] Recovery completed
Sep 09 16:08:44.944278 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.944261    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 16:08:44.946031 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.946001    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-81.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 16:08:44.946104 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.946046    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-81.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 16:08:44.946104 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.946059    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-81.ec2.internal" event="NodeHasSufficientPID"
Sep 09 16:08:44.946612 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.946595    2567 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 09 16:08:44.946612 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.946610    2567 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 09 16:08:44.946612 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.946614    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 16:08:44.946753 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.946626    2567 state_mem.go:36] "Initialized new in-memory state store"
Sep 09 16:08:44.947710 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.947692    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-81.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 16:08:44.947783 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.947725    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-81.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 16:08:44.947783 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.947736    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-81.ec2.internal" event="NodeHasSufficientPID"
Sep 09 16:08:44.947943 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:44.947860    2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-81.ec2.internal.18d3b2d9ab31c331  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-81.ec2.internal,UID:ip-10-0-128-81.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-81.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-81.ec2.internal,},FirstTimestamp:2026-09-09 16:08:44.946031409 +0000 UTC m=+0.363881399,LastTimestamp:2026-09-09 16:08:44.946031409 +0000 UTC m=+0.363881399,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-81.ec2.internal,}"
Sep 09 16:08:44.948845 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.948832    2567 policy_none.go:49] "None policy: Start"
Sep 09 16:08:44.948884 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.948848    2567 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 09 16:08:44.948884 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.948858    2567 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 09 16:08:44.950935 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.950923    2567 policy_none.go:47] "Start"
Sep 09 16:08:44.956068 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:44.956007    2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-81.ec2.internal.18d3b2d9ab321950  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-81.ec2.internal,UID:ip-10-0-128-81.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-128-81.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-128-81.ec2.internal,},FirstTimestamp:2026-09-09 16:08:44.946053456 +0000 UTC m=+0.363903445,LastTimestamp:2026-09-09 16:08:44.946053456 +0000 UTC m=+0.363903445,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-81.ec2.internal,}"
Sep 09 16:08:44.963827 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:44.963752    2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-81.ec2.internal.18d3b2d9ab324082  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-81.ec2.internal,UID:ip-10-0-128-81.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-128-81.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-128-81.ec2.internal,},FirstTimestamp:2026-09-09 16:08:44.94606349 +0000 UTC m=+0.363913478,LastTimestamp:2026-09-09 16:08:44.94606349 +0000 UTC m=+0.363913478,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-81.ec2.internal,}"
Sep 09 16:08:44.969169 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.969146    2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-jpkbn"
Sep 09 16:08:44.974405 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:44.974339    2567 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-128-81.ec2.internal.18d3b2d9ab31c331\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-81.ec2.internal.18d3b2d9ab31c331  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-81.ec2.internal,UID:ip-10-0-128-81.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-81.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-81.ec2.internal,},FirstTimestamp:2026-09-09 16:08:44.946031409 +0000 UTC m=+0.363881399,LastTimestamp:2026-09-09 16:08:44.947710224 +0000 UTC m=+0.365560220,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-81.ec2.internal,}"
Sep 09 16:08:44.979036 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.979020    2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-jpkbn"
Sep 09 16:08:45.006326 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.990059    2567 manager.go:337] "Starting Device Plugin manager"
Sep 09 16:08:45.006326 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:44.990109    2567 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 09 16:08:45.006326 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.990122    2567 server.go:87] "Starting device plugin registration server"
Sep 09 16:08:45.006326 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.990432    2567 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 09 16:08:45.006326 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.990587    2567 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 09 16:08:45.006326 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.990689    2567 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 09 16:08:45.006326 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.990768    2567 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 09 16:08:45.006326 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:44.990777    2567 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 09 16:08:45.006326 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:44.991286    2567 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 09 16:08:45.006326 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:44.991332    2567 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-81.ec2.internal\" not found"
Sep 09 16:08:45.028011 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.027976    2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 09 16:08:45.029228 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.029214    2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 09 16:08:45.029293 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.029260    2567 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 09 16:08:45.029293 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.029282    2567 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 09 16:08:45.029365 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:45.029318    2567 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 09 16:08:45.046407 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.046385    2567 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 16:08:45.091552 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.091491    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 16:08:45.092401 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.092385    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-81.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 16:08:45.092505 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.092415    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-81.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 16:08:45.092505 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.092426    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-81.ec2.internal" event="NodeHasSufficientPID"
Sep 09 16:08:45.092505 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.092449    2567 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-81.ec2.internal"
Sep 09 16:08:45.101274 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.101259    2567 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-81.ec2.internal"
Sep 09 16:08:45.101350 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:45.101279    2567 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-81.ec2.internal\": node \"ip-10-0-128-81.ec2.internal\" not found"
Sep 09 16:08:45.127313 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:45.127293    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-81.ec2.internal\" not found"
Sep 09 16:08:45.130366 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.130351    2567 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-81.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-81.ec2.internal"]
Sep 09 16:08:45.130438 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.130428    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 16:08:45.131903 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.131886    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-81.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 16:08:45.131981 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.131914    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-81.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 16:08:45.131981 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.131928    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-81.ec2.internal" event="NodeHasSufficientPID"
Sep 09 16:08:45.134341 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.134326    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 16:08:45.134487 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.134471    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-81.ec2.internal"
Sep 09 16:08:45.134543 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.134509    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 16:08:45.135052 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.135030    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-81.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 16:08:45.135144 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.135034    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-81.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 16:08:45.135144 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.135070    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-81.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 16:08:45.135144 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.135081    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-81.ec2.internal" event="NodeHasSufficientPID"
Sep 09 16:08:45.135144 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.135091    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-81.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 16:08:45.135144 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.135103    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-81.ec2.internal" event="NodeHasSufficientPID"
Sep 09 16:08:45.137362 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.137344    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-81.ec2.internal"
Sep 09 16:08:45.137484 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.137379    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 16:08:45.138079 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.138063    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-81.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 16:08:45.138156 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.138089    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-81.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 16:08:45.138156 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.138099    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-81.ec2.internal" event="NodeHasSufficientPID"
Sep 09 16:08:45.157022 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:45.157003    2567 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-81.ec2.internal\" not found" node="ip-10-0-128-81.ec2.internal"
Sep 09 16:08:45.161517 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:45.161502    2567 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-81.ec2.internal\" not found" node="ip-10-0-128-81.ec2.internal"
Sep 09 16:08:45.227143 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.227109    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/271bfa508befdbc16e7141fd83859452-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-81.ec2.internal\" (UID: \"271bfa508befdbc16e7141fd83859452\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-81.ec2.internal"
Sep 09 16:08:45.227143 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.227140    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fd9da504dd49d348bc16b790735150bc-config\") pod \"kube-apiserver-proxy-ip-10-0-128-81.ec2.internal\" (UID: \"fd9da504dd49d348bc16b790735150bc\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-81.ec2.internal"
Sep 09 16:08:45.227345 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.227157    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/271bfa508befdbc16e7141fd83859452-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-81.ec2.internal\" (UID: \"271bfa508befdbc16e7141fd83859452\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-81.ec2.internal"
Sep 09 16:08:45.228164 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:45.228150    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-81.ec2.internal\" not found"
Sep 09 16:08:45.327804 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.327769    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fd9da504dd49d348bc16b790735150bc-config\") pod \"kube-apiserver-proxy-ip-10-0-128-81.ec2.internal\" (UID: \"fd9da504dd49d348bc16b790735150bc\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-81.ec2.internal"
Sep 09 16:08:45.327939 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.327818    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/271bfa508befdbc16e7141fd83859452-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-81.ec2.internal\" (UID: \"271bfa508befdbc16e7141fd83859452\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-81.ec2.internal"
Sep 09 16:08:45.327939 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.327834    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fd9da504dd49d348bc16b790735150bc-config\") pod \"kube-apiserver-proxy-ip-10-0-128-81.ec2.internal\" (UID: \"fd9da504dd49d348bc16b790735150bc\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-81.ec2.internal"
Sep 09 16:08:45.327939 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.327840    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/271bfa508befdbc16e7141fd83859452-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-81.ec2.internal\" (UID: \"271bfa508befdbc16e7141fd83859452\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-81.ec2.internal"
Sep 09 16:08:45.327939 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.327888    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/271bfa508befdbc16e7141fd83859452-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-81.ec2.internal\" (UID: \"271bfa508befdbc16e7141fd83859452\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-81.ec2.internal"
Sep 09 16:08:45.327939 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.327868    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/271bfa508befdbc16e7141fd83859452-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-81.ec2.internal\" (UID: \"271bfa508befdbc16e7141fd83859452\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-81.ec2.internal"
Sep 09 16:08:45.328823 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:45.328807    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-81.ec2.internal\" not found"
Sep 09 16:08:45.429600 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:45.429516    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-81.ec2.internal\" not found"
Sep 09 16:08:45.459721 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.459689    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-81.ec2.internal"
Sep 09 16:08:45.464900 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.464797    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-81.ec2.internal"
Sep 09 16:08:45.530040 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:45.530006    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-81.ec2.internal\" not found"
Sep 09 16:08:45.630500 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:45.630469    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-81.ec2.internal\" not found"
Sep 09 16:08:45.730930 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:45.730865    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-81.ec2.internal\" not found"
Sep 09 16:08:45.831315 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:45.831276    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-81.ec2.internal\" not found"
Sep 09 16:08:45.860470 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.860429    2567 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 09 16:08:45.860887 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.860604    2567 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 09 16:08:45.888301 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.888275    2567 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 16:08:45.903956 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.903933    2567 apiserver.go:52] "Watching apiserver"
Sep 09 16:08:45.920470 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.919966    2567 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 09 16:08:45.922952 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.922926    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8hqn","openshift-dns/node-resolver-cnvzz","openshift-network-diagnostics/network-check-target-p4sh4","openshift-ovn-kubernetes/ovnkube-node-zrj9v","kube-system/konnectivity-agent-m7xl5","openshift-cluster-node-tuning-operator/tuned-rbvgk","openshift-image-registry/node-ca-c7kvc","openshift-multus/multus-additional-cni-plugins-l82fl","openshift-multus/multus-q2pgn","openshift-multus/network-metrics-daemon-t29nl","openshift-network-operator/iptables-alerter-jl8xm","kube-system/global-pull-secret-syncer-wk4dc"]
Sep 09 16:08:45.923981 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.923967    2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 09 16:08:45.925300 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.925286    2567 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-81.ec2.internal"
Sep 09 16:08:45.925467 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.925440    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8hqn"
Sep 09 16:08:45.927696 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.927672    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-cnvzz"
Sep 09 16:08:45.927696 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.927686    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 09 16:08:45.927858 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.927804    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 09 16:08:45.927967 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.927952    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 09 16:08:45.928253 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.928240    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-sgk8s\""
Sep 09 16:08:45.929634 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.929617    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 09 16:08:45.929851 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.929831    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-gccmf\""
Sep 09 16:08:45.929908 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.929849    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p4sh4"
Sep 09 16:08:45.929952 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:45.929924    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p4sh4" podUID="3ef88440-a7bd-4a04-8293-b8121e14655e"
Sep 09 16:08:45.930035 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.930020    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 09 16:08:45.930638 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.930623    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/21b4fd71-6273-4029-889d-aa66470c26c1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-f8hqn\" (UID: \"21b4fd71-6273-4029-889d-aa66470c26c1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8hqn"
Sep 09 16:08:45.930679 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.930645    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/21b4fd71-6273-4029-889d-aa66470c26c1-etc-selinux\") pod \"aws-ebs-csi-driver-node-f8hqn\" (UID: \"21b4fd71-6273-4029-889d-aa66470c26c1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8hqn"
Sep 09 16:08:45.930679 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.930665    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/21b4fd71-6273-4029-889d-aa66470c26c1-sys-fs\") pod \"aws-ebs-csi-driver-node-f8hqn\" (UID: \"21b4fd71-6273-4029-889d-aa66470c26c1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8hqn"
Sep 09 16:08:45.930749 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.930679    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/deb44589-97aa-464f-95b0-61362f120948-hosts-file\") pod \"node-resolver-cnvzz\" (UID: \"deb44589-97aa-464f-95b0-61362f120948\") " pod="openshift-dns/node-resolver-cnvzz"
Sep 09 16:08:45.930749 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.930693    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/21b4fd71-6273-4029-889d-aa66470c26c1-socket-dir\") pod \"aws-ebs-csi-driver-node-f8hqn\" (UID: \"21b4fd71-6273-4029-889d-aa66470c26c1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8hqn"
Sep 09 16:08:45.930749 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.930707    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/21b4fd71-6273-4029-889d-aa66470c26c1-registration-dir\") pod \"aws-ebs-csi-driver-node-f8hqn\" (UID: \"21b4fd71-6273-4029-889d-aa66470c26c1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8hqn"
Sep 09 16:08:45.930749 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.930720    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/21b4fd71-6273-4029-889d-aa66470c26c1-device-dir\") pod \"aws-ebs-csi-driver-node-f8hqn\" (UID: \"21b4fd71-6273-4029-889d-aa66470c26c1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8hqn"
Sep 09 16:08:45.930891 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.930751    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glbtk\" (UniqueName: \"kubernetes.io/projected/21b4fd71-6273-4029-889d-aa66470c26c1-kube-api-access-glbtk\") pod \"aws-ebs-csi-driver-node-f8hqn\" (UID: \"21b4fd71-6273-4029-889d-aa66470c26c1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8hqn"
Sep 09 16:08:45.930891 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.930778    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/deb44589-97aa-464f-95b0-61362f120948-tmp-dir\") pod \"node-resolver-cnvzz\" (UID: \"deb44589-97aa-464f-95b0-61362f120948\") " pod="openshift-dns/node-resolver-cnvzz"
Sep 09 16:08:45.930891 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.930796    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57p2n\" (UniqueName: \"kubernetes.io/projected/deb44589-97aa-464f-95b0-61362f120948-kube-api-access-57p2n\") pod \"node-resolver-cnvzz\" (UID: \"deb44589-97aa-464f-95b0-61362f120948\") " pod="openshift-dns/node-resolver-cnvzz"
Sep 09 16:08:45.932114 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.932098    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v"
Sep 09 16:08:45.934187 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.934169    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 09 16:08:45.934312 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.934295    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-m7xl5"
Sep 09 16:08:45.934354 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.934341    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 09 16:08:45.934398 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.934351    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-9f7xv\""
Sep 09 16:08:45.934558 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.934538    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 09 16:08:45.934627 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.934567    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 09 16:08:45.934627 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.934582    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 09 16:08:45.935486 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.935473    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 09 16:08:45.936230 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.936214    2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 09 16:08:45.936315 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.936273    2567 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-81.ec2.internal"
Sep 09 16:08:45.936727 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.936710    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rbvgk"
Sep 09 16:08:45.937079 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.937065    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 09 16:08:45.937161 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.937148    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-p5sjg\""
Sep 09 16:08:45.937198 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.937159    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 09 16:08:45.937841 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.937826    2567 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 09 16:08:45.938806 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.938786    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 09 16:08:45.938878 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.938863    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-hxbbt\""
Sep 09 16:08:45.938948 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.938888    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 09 16:08:45.938948 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.938937    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-c7kvc"
Sep 09 16:08:45.941007 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.940994    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 09 16:08:45.941081 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.941011    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 09 16:08:45.941081 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.941039    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-997nh\""
Sep 09 16:08:45.941171 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.941011    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 09 16:08:45.941260 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.941245    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-l82fl"
Sep 09 16:08:45.943367 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.943351    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 09 16:08:45.943367 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.943371    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-2l99j\""
Sep 09 16:08:45.943541 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.943358    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 09 16:08:45.943541 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.943475    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-q2pgn"
Sep 09 16:08:45.943829 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.943814    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 09 16:08:45.943868 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.943855    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 09 16:08:45.943910 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.943814    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 09 16:08:45.944339 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.944325    2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 09 16:08:45.945773 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.945757    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-2gclv\""
Sep 09 16:08:45.945868 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.945774    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t29nl"
Sep 09 16:08:45.945868 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:45.945825    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t29nl" podUID="73b8ff58-aa37-4fab-9632-9b0f5d409902"
Sep 09 16:08:45.946134 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.946118    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 09 16:08:45.948143 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.948125    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-jl8xm"
Sep 09 16:08:45.950141 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.950121    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 09 16:08:45.950275 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.950258    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-7gmbr\""
Sep 09 16:08:45.950425 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.950394    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-81.ec2.internal"]
Sep 09 16:08:45.950501 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.950424    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 09 16:08:45.950501 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.950443    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-128-81.ec2.internal"]
Sep 09 16:08:45.950501 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.950480    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wk4dc"
Sep 09 16:08:45.950653 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.950519    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 09 16:08:45.950653 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:45.950533    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wk4dc" podUID="28f47047-e391-44cb-836e-df79db9e4057"
Sep 09 16:08:45.960228 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.958712    2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-fqq4n"
Sep 09 16:08:45.965943 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.965925    2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-fqq4n"
Sep 09 16:08:45.982931 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.982859    2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-08 16:03:45 +0000 UTC" deadline="2028-03-14 11:05:19.683883195 +0000 UTC"
Sep 09 16:08:45.982931 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:45.982903    2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13242h56m33.700982346s"
Sep 09 16:08:46.026975 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.026958    2567 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 09 16:08:46.031031 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.031009    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/28f47047-e391-44cb-836e-df79db9e4057-dbus\") pod \"global-pull-secret-syncer-wk4dc\" (UID: \"28f47047-e391-44cb-836e-df79db9e4057\") " pod="kube-system/global-pull-secret-syncer-wk4dc"
Sep 09 16:08:46.031135 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.031048    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57p2n\" (UniqueName: \"kubernetes.io/projected/deb44589-97aa-464f-95b0-61362f120948-kube-api-access-57p2n\") pod \"node-resolver-cnvzz\" (UID: \"deb44589-97aa-464f-95b0-61362f120948\") " pod="openshift-dns/node-resolver-cnvzz"
Sep 09 16:08:46.031135 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.031077    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/15f41781-3a63-422e-9355-e37c98cc7c07-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-l82fl\" (UID: \"15f41781-3a63-422e-9355-e37c98cc7c07\") " pod="openshift-multus/multus-additional-cni-plugins-l82fl"
Sep 09 16:08:46.031246 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.031186    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7rrh\" (UniqueName: \"kubernetes.io/projected/99b48550-bc12-4084-bd6d-eb76f2311382-kube-api-access-h7rrh\") pod \"ovnkube-node-zrj9v\" (UID: \"99b48550-bc12-4084-bd6d-eb76f2311382\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v"
Sep 09 16:08:46.031246 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.031203    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f996fce2-faf6-474a-997d-8a9419e788d8-serviceca\") pod \"node-ca-c7kvc\" (UID: \"f996fce2-faf6-474a-997d-8a9419e788d8\") " pod="openshift-image-registry/node-ca-c7kvc"
Sep 09 16:08:46.031246 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.031223    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/36d2c95e-5f46-413b-a6f5-4f0c4bd57b05-multus-cni-dir\") pod \"multus-q2pgn\" (UID: \"36d2c95e-5f46-413b-a6f5-4f0c4bd57b05\") " pod="openshift-multus/multus-q2pgn"
Sep 09 16:08:46.031396 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.031267    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qq4nj\" (UniqueName: \"kubernetes.io/projected/f996fce2-faf6-474a-997d-8a9419e788d8-kube-api-access-qq4nj\") pod \"node-ca-c7kvc\" (UID: \"f996fce2-faf6-474a-997d-8a9419e788d8\") " pod="openshift-image-registry/node-ca-c7kvc"
Sep 09 16:08:46.031396 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.031300    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvkr2\" (UniqueName: \"kubernetes.io/projected/73b8ff58-aa37-4fab-9632-9b0f5d409902-kube-api-access-tvkr2\") pod \"network-metrics-daemon-t29nl\" (UID: \"73b8ff58-aa37-4fab-9632-9b0f5d409902\") " pod="openshift-multus/network-metrics-daemon-t29nl"
Sep 09 16:08:46.031396 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.031321    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/83598a1c-9f92-44aa-afa9-42556f6c0127-agent-certs\") pod \"konnectivity-agent-m7xl5\" (UID: \"83598a1c-9f92-44aa-afa9-42556f6c0127\") " pod="kube-system/konnectivity-agent-m7xl5"
Sep 09 16:08:46.031396 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.031345    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/15f41781-3a63-422e-9355-e37c98cc7c07-cnibin\") pod \"multus-additional-cni-plugins-l82fl\" (UID: \"15f41781-3a63-422e-9355-e37c98cc7c07\") " pod="openshift-multus/multus-additional-cni-plugins-l82fl"
Sep 09 16:08:46.031396 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.031368    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8b9j\" (UniqueName: \"kubernetes.io/projected/15f41781-3a63-422e-9355-e37c98cc7c07-kube-api-access-s8b9j\") pod \"multus-additional-cni-plugins-l82fl\" (UID: \"15f41781-3a63-422e-9355-e37c98cc7c07\") " pod="openshift-multus/multus-additional-cni-plugins-l82fl"
Sep 09 16:08:46.031712 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.031425    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/36d2c95e-5f46-413b-a6f5-4f0c4bd57b05-host-var-lib-kubelet\") pod \"multus-q2pgn\" (UID: \"36d2c95e-5f46-413b-a6f5-4f0c4bd57b05\") " pod="openshift-multus/multus-q2pgn"
Sep 09 16:08:46.031712 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.031475    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v872q\" (UniqueName: \"kubernetes.io/projected/36d2c95e-5f46-413b-a6f5-4f0c4bd57b05-kube-api-access-v872q\") pod \"multus-q2pgn\" (UID: \"36d2c95e-5f46-413b-a6f5-4f0c4bd57b05\") " pod="openshift-multus/multus-q2pgn"
Sep 09 16:08:46.031712 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.031505    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/99b48550-bc12-4084-bd6d-eb76f2311382-host-run-netns\") pod \"ovnkube-node-zrj9v\" (UID: \"99b48550-bc12-4084-bd6d-eb76f2311382\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v"
Sep 09 16:08:46.031712 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.031542    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/99b48550-bc12-4084-bd6d-eb76f2311382-run-openvswitch\") pod \"ovnkube-node-zrj9v\" (UID: \"99b48550-bc12-4084-bd6d-eb76f2311382\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v"
Sep 09 16:08:46.031712 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.031565    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/99b48550-bc12-4084-bd6d-eb76f2311382-run-ovn\") pod \"ovnkube-node-zrj9v\" (UID: \"99b48550-bc12-4084-bd6d-eb76f2311382\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v"
Sep 09 16:08:46.031712 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.031594    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/21b4fd71-6273-4029-889d-aa66470c26c1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-f8hqn\" (UID: \"21b4fd71-6273-4029-889d-aa66470c26c1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8hqn"
Sep 09 16:08:46.031712 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.031632    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/99b48550-bc12-4084-bd6d-eb76f2311382-host-kubelet\") pod \"ovnkube-node-zrj9v\" (UID: \"99b48550-bc12-4084-bd6d-eb76f2311382\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v"
Sep 09 16:08:46.031712 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.031664    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/99b48550-bc12-4084-bd6d-eb76f2311382-var-lib-openvswitch\") pod \"ovnkube-node-zrj9v\" (UID: \"99b48550-bc12-4084-bd6d-eb76f2311382\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v"
Sep 09 16:08:46.031712 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.031688    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/99b48550-bc12-4084-bd6d-eb76f2311382-log-socket\") pod \"ovnkube-node-zrj9v\" (UID: \"99b48550-bc12-4084-bd6d-eb76f2311382\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v"
Sep 09 16:08:46.031712 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.031665    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/21b4fd71-6273-4029-889d-aa66470c26c1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-f8hqn\" (UID: \"21b4fd71-6273-4029-889d-aa66470c26c1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8hqn"
Sep 09 16:08:46.031712 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.031713    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j42zn\" (UniqueName: \"kubernetes.io/projected/43ae6ccf-db13-471c-a336-733bbea3e930-kube-api-access-j42zn\") pod \"iptables-alerter-jl8xm\" (UID: \"43ae6ccf-db13-471c-a336-733bbea3e930\") " pod="openshift-network-operator/iptables-alerter-jl8xm"
Sep 09 16:08:46.032129 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.031734    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/28f47047-e391-44cb-836e-df79db9e4057-kubelet-config\") pod \"global-pull-secret-syncer-wk4dc\" (UID: \"28f47047-e391-44cb-836e-df79db9e4057\") " pod="kube-system/global-pull-secret-syncer-wk4dc"
Sep 09 16:08:46.032129 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.031756    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/28f47047-e391-44cb-836e-df79db9e4057-original-pull-secret\") pod \"global-pull-secret-syncer-wk4dc\" (UID: \"28f47047-e391-44cb-836e-df79db9e4057\") " pod="kube-system/global-pull-secret-syncer-wk4dc"
Sep 09 16:08:46.032129 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.031780    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/17c51357-18e5-4f81-bb5b-9c73e38e55bc-etc-kubernetes\") pod \"tuned-rbvgk\" (UID: \"17c51357-18e5-4f81-bb5b-9c73e38e55bc\") " pod="openshift-cluster-node-tuning-operator/tuned-rbvgk"
Sep 09 16:08:46.032129 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.031803    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/15f41781-3a63-422e-9355-e37c98cc7c07-cni-binary-copy\") pod \"multus-additional-cni-plugins-l82fl\" (UID: \"15f41781-3a63-422e-9355-e37c98cc7c07\") " pod="openshift-multus/multus-additional-cni-plugins-l82fl"
Sep 09 16:08:46.032129 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.031825    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/36d2c95e-5f46-413b-a6f5-4f0c4bd57b05-host-run-multus-certs\") pod \"multus-q2pgn\" (UID: \"36d2c95e-5f46-413b-a6f5-4f0c4bd57b05\") " pod="openshift-multus/multus-q2pgn"
Sep 09 16:08:46.032129 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.031845    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/17c51357-18e5-4f81-bb5b-9c73e38e55bc-etc-sysctl-conf\") pod \"tuned-rbvgk\" (UID: \"17c51357-18e5-4f81-bb5b-9c73e38e55bc\") " pod="openshift-cluster-node-tuning-operator/tuned-rbvgk"
Sep 09 16:08:46.032129 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.031867    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/17c51357-18e5-4f81-bb5b-9c73e38e55bc-etc-systemd\") pod \"tuned-rbvgk\" (UID: \"17c51357-18e5-4f81-bb5b-9c73e38e55bc\") " pod="openshift-cluster-node-tuning-operator/tuned-rbvgk"
Sep 09 16:08:46.032129 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.031889    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/36d2c95e-5f46-413b-a6f5-4f0c4bd57b05-host-run-netns\") pod \"multus-q2pgn\" (UID: \"36d2c95e-5f46-413b-a6f5-4f0c4bd57b05\") " pod="openshift-multus/multus-q2pgn"
Sep 09 16:08:46.032129 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.031911    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/99b48550-bc12-4084-bd6d-eb76f2311382-run-systemd\") pod \"ovnkube-node-zrj9v\" (UID: \"99b48550-bc12-4084-bd6d-eb76f2311382\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v"
Sep 09 16:08:46.032129 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.031931    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/99b48550-bc12-4084-bd6d-eb76f2311382-node-log\") pod \"ovnkube-node-zrj9v\" (UID: \"99b48550-bc12-4084-bd6d-eb76f2311382\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v"
Sep 09 16:08:46.032129 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.031957    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/21b4fd71-6273-4029-889d-aa66470c26c1-socket-dir\") pod \"aws-ebs-csi-driver-node-f8hqn\" (UID: \"21b4fd71-6273-4029-889d-aa66470c26c1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8hqn"
Sep 09 16:08:46.032129 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.031979    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/83598a1c-9f92-44aa-afa9-42556f6c0127-konnectivity-ca\") pod \"konnectivity-agent-m7xl5\" (UID: \"83598a1c-9f92-44aa-afa9-42556f6c0127\") " pod="kube-system/konnectivity-agent-m7xl5"
Sep 09 16:08:46.032129 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.032001    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/36d2c95e-5f46-413b-a6f5-4f0c4bd57b05-system-cni-dir\") pod \"multus-q2pgn\" (UID: \"36d2c95e-5f46-413b-a6f5-4f0c4bd57b05\") " pod="openshift-multus/multus-q2pgn"
Sep 09 16:08:46.032129 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.032022    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/36d2c95e-5f46-413b-a6f5-4f0c4bd57b05-cnibin\") pod \"multus-q2pgn\" (UID: \"36d2c95e-5f46-413b-a6f5-4f0c4bd57b05\") " pod="openshift-multus/multus-q2pgn"
Sep 09 16:08:46.032129 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.032050    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/99b48550-bc12-4084-bd6d-eb76f2311382-ovnkube-script-lib\") pod \"ovnkube-node-zrj9v\" (UID: \"99b48550-bc12-4084-bd6d-eb76f2311382\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v"
Sep 09 16:08:46.032129 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.032078    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/15f41781-3a63-422e-9355-e37c98cc7c07-system-cni-dir\") pod \"multus-additional-cni-plugins-l82fl\" (UID: \"15f41781-3a63-422e-9355-e37c98cc7c07\") " pod="openshift-multus/multus-additional-cni-plugins-l82fl"
Sep 09 16:08:46.032806 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.032093    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhw55\" (UniqueName: \"kubernetes.io/projected/3ef88440-a7bd-4a04-8293-b8121e14655e-kube-api-access-mhw55\") pod \"network-check-target-p4sh4\" (UID: \"3ef88440-a7bd-4a04-8293-b8121e14655e\") " pod="openshift-network-diagnostics/network-check-target-p4sh4"
Sep 09 16:08:46.032806 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.032114    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/21b4fd71-6273-4029-889d-aa66470c26c1-socket-dir\") pod \"aws-ebs-csi-driver-node-f8hqn\" (UID: \"21b4fd71-6273-4029-889d-aa66470c26c1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8hqn"
Sep 09 16:08:46.032806 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.032129    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/17c51357-18e5-4f81-bb5b-9c73e38e55bc-var-lib-kubelet\") pod \"tuned-rbvgk\" (UID: \"17c51357-18e5-4f81-bb5b-9c73e38e55bc\") " pod="openshift-cluster-node-tuning-operator/tuned-rbvgk"
Sep 09 16:08:46.032806 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.032173    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/17c51357-18e5-4f81-bb5b-9c73e38e55bc-etc-tuned\") pod \"tuned-rbvgk\" (UID: \"17c51357-18e5-4f81-bb5b-9c73e38e55bc\") " pod="openshift-cluster-node-tuning-operator/tuned-rbvgk"
Sep 09 16:08:46.032806 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.032204    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/deb44589-97aa-464f-95b0-61362f120948-tmp-dir\") pod \"node-resolver-cnvzz\" (UID: \"deb44589-97aa-464f-95b0-61362f120948\") " pod="openshift-dns/node-resolver-cnvzz"
Sep 09 16:08:46.032806 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.032230    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/17c51357-18e5-4f81-bb5b-9c73e38e55bc-lib-modules\") pod \"tuned-rbvgk\" (UID: \"17c51357-18e5-4f81-bb5b-9c73e38e55bc\") " pod="openshift-cluster-node-tuning-operator/tuned-rbvgk"
Sep 09 16:08:46.032806 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.032253    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hbkc\" (UniqueName: \"kubernetes.io/projected/17c51357-18e5-4f81-bb5b-9c73e38e55bc-kube-api-access-6hbkc\") pod \"tuned-rbvgk\" (UID: \"17c51357-18e5-4f81-bb5b-9c73e38e55bc\") " pod="openshift-cluster-node-tuning-operator/tuned-rbvgk"
Sep 09 16:08:46.032806 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.032283    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/17c51357-18e5-4f81-bb5b-9c73e38e55bc-etc-modprobe-d\") pod \"tuned-rbvgk\" (UID: \"17c51357-18e5-4f81-bb5b-9c73e38e55bc\") " pod="openshift-cluster-node-tuning-operator/tuned-rbvgk"
Sep 09 16:08:46.032806 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.032305    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/17c51357-18e5-4f81-bb5b-9c73e38e55bc-run\") pod \"tuned-rbvgk\" (UID: \"17c51357-18e5-4f81-bb5b-9c73e38e55bc\") " pod="openshift-cluster-node-tuning-operator/tuned-rbvgk"
Sep 09 16:08:46.032806 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.032341    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/43ae6ccf-db13-471c-a336-733bbea3e930-iptables-alerter-script\") pod \"iptables-alerter-jl8xm\" (UID: \"43ae6ccf-db13-471c-a336-733bbea3e930\") " pod="openshift-network-operator/iptables-alerter-jl8xm"
Sep 09 16:08:46.032806 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.032366    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/17c51357-18e5-4f81-bb5b-9c73e38e55bc-etc-sysconfig\") pod \"tuned-rbvgk\" (UID: \"17c51357-18e5-4f81-bb5b-9c73e38e55bc\") " pod="openshift-cluster-node-tuning-operator/tuned-rbvgk"
Sep 09 16:08:46.032806 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.032388    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/17c51357-18e5-4f81-bb5b-9c73e38e55bc-sys\") pod \"tuned-rbvgk\" (UID: \"17c51357-18e5-4f81-bb5b-9c73e38e55bc\") " pod="openshift-cluster-node-tuning-operator/tuned-rbvgk"
Sep 09 16:08:46.032806 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.032413    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/15f41781-3a63-422e-9355-e37c98cc7c07-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-l82fl\" (UID: \"15f41781-3a63-422e-9355-e37c98cc7c07\") " pod="openshift-multus/multus-additional-cni-plugins-l82fl"
Sep 09 16:08:46.032806 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.032448    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/99b48550-bc12-4084-bd6d-eb76f2311382-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zrj9v\" (UID: \"99b48550-bc12-4084-bd6d-eb76f2311382\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v"
Sep 09 16:08:46.032806 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.032494    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/99b48550-bc12-4084-bd6d-eb76f2311382-env-overrides\") pod \"ovnkube-node-zrj9v\" (UID: \"99b48550-bc12-4084-bd6d-eb76f2311382\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v"
Sep 09 16:08:46.032806 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.032514    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/21b4fd71-6273-4029-889d-aa66470c26c1-etc-selinux\") pod \"aws-ebs-csi-driver-node-f8hqn\" (UID: \"21b4fd71-6273-4029-889d-aa66470c26c1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8hqn"
Sep 09 16:08:46.033417 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.032546    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/deb44589-97aa-464f-95b0-61362f120948-hosts-file\") pod \"node-resolver-cnvzz\" (UID: \"deb44589-97aa-464f-95b0-61362f120948\") " pod="openshift-dns/node-resolver-cnvzz"
Sep 09 16:08:46.033417 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.032595    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/deb44589-97aa-464f-95b0-61362f120948-hosts-file\") pod \"node-resolver-cnvzz\" (UID: \"deb44589-97aa-464f-95b0-61362f120948\") " pod="openshift-dns/node-resolver-cnvzz"
Sep 09 16:08:46.033417 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.032627    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/21b4fd71-6273-4029-889d-aa66470c26c1-etc-selinux\") pod \"aws-ebs-csi-driver-node-f8hqn\" (UID: \"21b4fd71-6273-4029-889d-aa66470c26c1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8hqn"
Sep 09 16:08:46.033417 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.032638    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/99b48550-bc12-4084-bd6d-eb76f2311382-host-cni-bin\") pod \"ovnkube-node-zrj9v\" (UID: \"99b48550-bc12-4084-bd6d-eb76f2311382\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v"
Sep 09 16:08:46.033417 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.032669    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/99b48550-bc12-4084-bd6d-eb76f2311382-host-cni-netd\") pod \"ovnkube-node-zrj9v\" (UID: \"99b48550-bc12-4084-bd6d-eb76f2311382\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v"
Sep 09 16:08:46.033417 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.032700    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/99b48550-bc12-4084-bd6d-eb76f2311382-ovn-node-metrics-cert\") pod \"ovnkube-node-zrj9v\" (UID: \"99b48550-bc12-4084-bd6d-eb76f2311382\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v"
Sep 09 16:08:46.033417 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.032721    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73b8ff58-aa37-4fab-9632-9b0f5d409902-metrics-certs\") pod \"network-metrics-daemon-t29nl\" (UID: \"73b8ff58-aa37-4fab-9632-9b0f5d409902\") " pod="openshift-multus/network-metrics-daemon-t29nl"
Sep 09 16:08:46.033417 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.032736    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/17c51357-18e5-4f81-bb5b-9c73e38e55bc-tmp\") pod \"tuned-rbvgk\" (UID: \"17c51357-18e5-4f81-bb5b-9c73e38e55bc\") " pod="openshift-cluster-node-tuning-operator/tuned-rbvgk"
Sep 09 16:08:46.033417 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.032753    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/21b4fd71-6273-4029-889d-aa66470c26c1-registration-dir\") pod \"aws-ebs-csi-driver-node-f8hqn\" (UID: \"21b4fd71-6273-4029-889d-aa66470c26c1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8hqn"
Sep 09 16:08:46.033417 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.032768    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/21b4fd71-6273-4029-889d-aa66470c26c1-device-dir\") pod \"aws-ebs-csi-driver-node-f8hqn\" (UID: \"21b4fd71-6273-4029-889d-aa66470c26c1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8hqn"
Sep 09 16:08:46.033417 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.032789    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/17c51357-18e5-4f81-bb5b-9c73e38e55bc-etc-sysctl-d\") pod \"tuned-rbvgk\" (UID: \"17c51357-18e5-4f81-bb5b-9c73e38e55bc\") " pod="openshift-cluster-node-tuning-operator/tuned-rbvgk"
Sep 09 16:08:46.033417 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.032804    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/36d2c95e-5f46-413b-a6f5-4f0c4bd57b05-cni-binary-copy\") pod \"multus-q2pgn\" (UID: \"36d2c95e-5f46-413b-a6f5-4f0c4bd57b05\") " pod="openshift-multus/multus-q2pgn"
Sep 09 16:08:46.033417 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.032818    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/21b4fd71-6273-4029-889d-aa66470c26c1-registration-dir\") pod \"aws-ebs-csi-driver-node-f8hqn\" (UID: \"21b4fd71-6273-4029-889d-aa66470c26c1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8hqn"
Sep 09 16:08:46.033417 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.032828    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/36d2c95e-5f46-413b-a6f5-4f0c4bd57b05-multus-socket-dir-parent\") pod \"multus-q2pgn\" (UID: \"36d2c95e-5f46-413b-a6f5-4f0c4bd57b05\") " pod="openshift-multus/multus-q2pgn"
Sep 09 16:08:46.033417 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.032862    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/21b4fd71-6273-4029-889d-aa66470c26c1-device-dir\") pod \"aws-ebs-csi-driver-node-f8hqn\" (UID: \"21b4fd71-6273-4029-889d-aa66470c26c1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8hqn"
Sep 09 16:08:46.033417 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.032872    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/99b48550-bc12-4084-bd6d-eb76f2311382-ovnkube-config\") pod \"ovnkube-node-zrj9v\" (UID: \"99b48550-bc12-4084-bd6d-eb76f2311382\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v"
Sep 09 16:08:46.033417 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.032896    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/99b48550-bc12-4084-bd6d-eb76f2311382-systemd-units\") pod \"ovnkube-node-zrj9v\" (UID: \"99b48550-bc12-4084-bd6d-eb76f2311382\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v"
Sep 09 16:08:46.033904 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.032917    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/99b48550-bc12-4084-bd6d-eb76f2311382-host-slash\") pod \"ovnkube-node-zrj9v\" (UID: \"99b48550-bc12-4084-bd6d-eb76f2311382\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v"
Sep 09 16:08:46.033904 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.032937    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/21b4fd71-6273-4029-889d-aa66470c26c1-sys-fs\") pod \"aws-ebs-csi-driver-node-f8hqn\" (UID: \"21b4fd71-6273-4029-889d-aa66470c26c1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8hqn"
Sep 09 16:08:46.033904 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.032989    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/21b4fd71-6273-4029-889d-aa66470c26c1-sys-fs\") pod \"aws-ebs-csi-driver-node-f8hqn\" (UID: \"21b4fd71-6273-4029-889d-aa66470c26c1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8hqn"
Sep 09 16:08:46.033904 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.032996    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/36d2c95e-5f46-413b-a6f5-4f0c4bd57b05-host-var-lib-cni-multus\") pod \"multus-q2pgn\" (UID: \"36d2c95e-5f46-413b-a6f5-4f0c4bd57b05\") " pod="openshift-multus/multus-q2pgn"
Sep 09 16:08:46.033904 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.033023    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/36d2c95e-5f46-413b-a6f5-4f0c4bd57b05-multus-conf-dir\") pod \"multus-q2pgn\" (UID: \"36d2c95e-5f46-413b-a6f5-4f0c4bd57b05\") " pod="openshift-multus/multus-q2pgn"
Sep 09 16:08:46.033904 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.033040    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/36d2c95e-5f46-413b-a6f5-4f0c4bd57b05-etc-kubernetes\") pod \"multus-q2pgn\" (UID: \"36d2c95e-5f46-413b-a6f5-4f0c4bd57b05\") " pod="openshift-multus/multus-q2pgn"
Sep 09 16:08:46.033904 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.033055    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/99b48550-bc12-4084-bd6d-eb76f2311382-host-run-ovn-kubernetes\") pod \"ovnkube-node-zrj9v\" (UID: \"99b48550-bc12-4084-bd6d-eb76f2311382\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v"
Sep 09 16:08:46.033904 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.033072    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glbtk\" (UniqueName: \"kubernetes.io/projected/21b4fd71-6273-4029-889d-aa66470c26c1-kube-api-access-glbtk\") pod \"aws-ebs-csi-driver-node-f8hqn\" (UID: \"21b4fd71-6273-4029-889d-aa66470c26c1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8hqn"
Sep 09 16:08:46.033904 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.033087    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/15f41781-3a63-422e-9355-e37c98cc7c07-tuning-conf-dir\") pod \"multus-additional-cni-plugins-l82fl\" (UID: \"15f41781-3a63-422e-9355-e37c98cc7c07\") " pod="openshift-multus/multus-additional-cni-plugins-l82fl"
Sep 09 16:08:46.033904 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.033100    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/36d2c95e-5f46-413b-a6f5-4f0c4bd57b05-hostroot\") pod \"multus-q2pgn\" (UID: \"36d2c95e-5f46-413b-a6f5-4f0c4bd57b05\") " pod="openshift-multus/multus-q2pgn"
Sep 09 16:08:46.033904 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.033133    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/36d2c95e-5f46-413b-a6f5-4f0c4bd57b05-multus-daemon-config\") pod \"multus-q2pgn\" (UID: \"36d2c95e-5f46-413b-a6f5-4f0c4bd57b05\") " pod="openshift-multus/multus-q2pgn"
Sep 09 16:08:46.033904 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.033221    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/99b48550-bc12-4084-bd6d-eb76f2311382-etc-openvswitch\") pod \"ovnkube-node-zrj9v\" (UID: \"99b48550-bc12-4084-bd6d-eb76f2311382\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v"
Sep 09 16:08:46.033904 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.033289    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f996fce2-faf6-474a-997d-8a9419e788d8-host\") pod \"node-ca-c7kvc\" (UID: \"f996fce2-faf6-474a-997d-8a9419e788d8\") " pod="openshift-image-registry/node-ca-c7kvc"
Sep 09 16:08:46.033904 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.033312    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/43ae6ccf-db13-471c-a336-733bbea3e930-host-slash\") pod \"iptables-alerter-jl8xm\" (UID: \"43ae6ccf-db13-471c-a336-733bbea3e930\") " pod="openshift-network-operator/iptables-alerter-jl8xm"
Sep 09 16:08:46.033904 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.033327    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/17c51357-18e5-4f81-bb5b-9c73e38e55bc-host\") pod \"tuned-rbvgk\" (UID: \"17c51357-18e5-4f81-bb5b-9c73e38e55bc\") " pod="openshift-cluster-node-tuning-operator/tuned-rbvgk"
Sep 09 16:08:46.033904 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.033349    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/36d2c95e-5f46-413b-a6f5-4f0c4bd57b05-host-run-k8s-cni-cncf-io\") pod \"multus-q2pgn\" (UID: \"36d2c95e-5f46-413b-a6f5-4f0c4bd57b05\") " pod="openshift-multus/multus-q2pgn"
Sep 09 16:08:46.033904 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.033372    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/36d2c95e-5f46-413b-a6f5-4f0c4bd57b05-host-var-lib-cni-bin\") pod \"multus-q2pgn\" (UID: \"36d2c95e-5f46-413b-a6f5-4f0c4bd57b05\") " pod="openshift-multus/multus-q2pgn"
Sep 09 16:08:46.034362 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.034094    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/deb44589-97aa-464f-95b0-61362f120948-tmp-dir\") pod \"node-resolver-cnvzz\" (UID: \"deb44589-97aa-464f-95b0-61362f120948\") " pod="openshift-dns/node-resolver-cnvzz"
Sep 09 16:08:46.036214 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.036196    2567 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 09 16:08:46.039854 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.039833    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-57p2n\" (UniqueName: \"kubernetes.io/projected/deb44589-97aa-464f-95b0-61362f120948-kube-api-access-57p2n\") pod \"node-resolver-cnvzz\" (UID: \"deb44589-97aa-464f-95b0-61362f120948\") " pod="openshift-dns/node-resolver-cnvzz"
Sep 09 16:08:46.040171 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.040155    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-glbtk\" (UniqueName: \"kubernetes.io/projected/21b4fd71-6273-4029-889d-aa66470c26c1-kube-api-access-glbtk\") pod \"aws-ebs-csi-driver-node-f8hqn\" (UID: \"21b4fd71-6273-4029-889d-aa66470c26c1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8hqn"
Sep 09 16:08:46.116908 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.116877    2567 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 16:08:46.133595 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.133572    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/15f41781-3a63-422e-9355-e37c98cc7c07-system-cni-dir\") pod \"multus-additional-cni-plugins-l82fl\" (UID: \"15f41781-3a63-422e-9355-e37c98cc7c07\") " pod="openshift-multus/multus-additional-cni-plugins-l82fl"
Sep 09 16:08:46.133744 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.133603    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhw55\" (UniqueName: \"kubernetes.io/projected/3ef88440-a7bd-4a04-8293-b8121e14655e-kube-api-access-mhw55\") pod \"network-check-target-p4sh4\" (UID: \"3ef88440-a7bd-4a04-8293-b8121e14655e\") " pod="openshift-network-diagnostics/network-check-target-p4sh4"
Sep 09 16:08:46.133744 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.133628    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/17c51357-18e5-4f81-bb5b-9c73e38e55bc-var-lib-kubelet\") pod \"tuned-rbvgk\" (UID: \"17c51357-18e5-4f81-bb5b-9c73e38e55bc\") " pod="openshift-cluster-node-tuning-operator/tuned-rbvgk"
Sep 09 16:08:46.133744 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.133651    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/17c51357-18e5-4f81-bb5b-9c73e38e55bc-etc-tuned\") pod \"tuned-rbvgk\" (UID: \"17c51357-18e5-4f81-bb5b-9c73e38e55bc\") " pod="openshift-cluster-node-tuning-operator/tuned-rbvgk"
Sep 09 16:08:46.133744 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.133646    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/15f41781-3a63-422e-9355-e37c98cc7c07-system-cni-dir\") pod \"multus-additional-cni-plugins-l82fl\" (UID: \"15f41781-3a63-422e-9355-e37c98cc7c07\") " pod="openshift-multus/multus-additional-cni-plugins-l82fl"
Sep 09 16:08:46.133744 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.133710    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/17c51357-18e5-4f81-bb5b-9c73e38e55bc-lib-modules\") pod \"tuned-rbvgk\" (UID: \"17c51357-18e5-4f81-bb5b-9c73e38e55bc\") " pod="openshift-cluster-node-tuning-operator/tuned-rbvgk"
Sep 09 16:08:46.133744 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.133732    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/17c51357-18e5-4f81-bb5b-9c73e38e55bc-var-lib-kubelet\") pod \"tuned-rbvgk\" (UID: \"17c51357-18e5-4f81-bb5b-9c73e38e55bc\") " pod="openshift-cluster-node-tuning-operator/tuned-rbvgk"
Sep 09 16:08:46.134023 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.133744    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hbkc\" (UniqueName: \"kubernetes.io/projected/17c51357-18e5-4f81-bb5b-9c73e38e55bc-kube-api-access-6hbkc\") pod \"tuned-rbvgk\" (UID: \"17c51357-18e5-4f81-bb5b-9c73e38e55bc\") " pod="openshift-cluster-node-tuning-operator/tuned-rbvgk"
Sep 09 16:08:46.134023 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.133774    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/17c51357-18e5-4f81-bb5b-9c73e38e55bc-etc-modprobe-d\") pod \"tuned-rbvgk\" (UID: \"17c51357-18e5-4f81-bb5b-9c73e38e55bc\") " pod="openshift-cluster-node-tuning-operator/tuned-rbvgk"
Sep 09 16:08:46.134023 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.133798    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/17c51357-18e5-4f81-bb5b-9c73e38e55bc-run\") pod \"tuned-rbvgk\" (UID: \"17c51357-18e5-4f81-bb5b-9c73e38e55bc\") " pod="openshift-cluster-node-tuning-operator/tuned-rbvgk"
Sep 09 16:08:46.134023 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.133822    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/43ae6ccf-db13-471c-a336-733bbea3e930-iptables-alerter-script\") pod \"iptables-alerter-jl8xm\" (UID: \"43ae6ccf-db13-471c-a336-733bbea3e930\") " pod="openshift-network-operator/iptables-alerter-jl8xm"
Sep 09 16:08:46.134023 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.133842    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/17c51357-18e5-4f81-bb5b-9c73e38e55bc-lib-modules\") pod \"tuned-rbvgk\" (UID: \"17c51357-18e5-4f81-bb5b-9c73e38e55bc\") " pod="openshift-cluster-node-tuning-operator/tuned-rbvgk"
Sep 09 16:08:46.134023 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.133845    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/17c51357-18e5-4f81-bb5b-9c73e38e55bc-etc-sysconfig\") pod \"tuned-rbvgk\" (UID: \"17c51357-18e5-4f81-bb5b-9c73e38e55bc\") " pod="openshift-cluster-node-tuning-operator/tuned-rbvgk"
Sep 09 16:08:46.134023 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.133918    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/17c51357-18e5-4f81-bb5b-9c73e38e55bc-sys\") pod \"tuned-rbvgk\" (UID: \"17c51357-18e5-4f81-bb5b-9c73e38e55bc\") " pod="openshift-cluster-node-tuning-operator/tuned-rbvgk"
Sep 09 16:08:46.134023 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.133934    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/17c51357-18e5-4f81-bb5b-9c73e38e55bc-run\") pod \"tuned-rbvgk\" (UID: \"17c51357-18e5-4f81-bb5b-9c73e38e55bc\") " pod="openshift-cluster-node-tuning-operator/tuned-rbvgk"
Sep 09 16:08:46.134023 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.133946    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/15f41781-3a63-422e-9355-e37c98cc7c07-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-l82fl\" (UID: \"15f41781-3a63-422e-9355-e37c98cc7c07\") " pod="openshift-multus/multus-additional-cni-plugins-l82fl"
Sep 09 16:08:46.134023 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.133958    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/17c51357-18e5-4f81-bb5b-9c73e38e55bc-sys\") pod \"tuned-rbvgk\" (UID: \"17c51357-18e5-4f81-bb5b-9c73e38e55bc\") " pod="openshift-cluster-node-tuning-operator/tuned-rbvgk"
Sep 09 16:08:46.134023 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.133919    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/17c51357-18e5-4f81-bb5b-9c73e38e55bc-etc-modprobe-d\") pod \"tuned-rbvgk\" (UID: \"17c51357-18e5-4f81-bb5b-9c73e38e55bc\") " pod="openshift-cluster-node-tuning-operator/tuned-rbvgk"
Sep 09 16:08:46.134023 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.133987    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/99b48550-bc12-4084-bd6d-eb76f2311382-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zrj9v\" (UID: \"99b48550-bc12-4084-bd6d-eb76f2311382\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v"
Sep 09 16:08:46.134023 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.134012    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/17c51357-18e5-4f81-bb5b-9c73e38e55bc-etc-sysconfig\") pod \"tuned-rbvgk\" (UID: \"17c51357-18e5-4f81-bb5b-9c73e38e55bc\") " pod="openshift-cluster-node-tuning-operator/tuned-rbvgk"
Sep 09 16:08:46.134023 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.134019    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/99b48550-bc12-4084-bd6d-eb76f2311382-env-overrides\") pod \"ovnkube-node-zrj9v\" (UID: \"99b48550-bc12-4084-bd6d-eb76f2311382\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v"
Sep 09 16:08:46.134862 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.134049    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/99b48550-bc12-4084-bd6d-eb76f2311382-host-cni-bin\") pod \"ovnkube-node-zrj9v\" (UID: \"99b48550-bc12-4084-bd6d-eb76f2311382\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v"
Sep 09 16:08:46.134862 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.134052    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/99b48550-bc12-4084-bd6d-eb76f2311382-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zrj9v\" (UID: \"99b48550-bc12-4084-bd6d-eb76f2311382\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v"
Sep 09 16:08:46.134862 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.134072    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/99b48550-bc12-4084-bd6d-eb76f2311382-host-cni-netd\") pod \"ovnkube-node-zrj9v\" (UID: \"99b48550-bc12-4084-bd6d-eb76f2311382\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v"
Sep 09 16:08:46.134862 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.134092    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/99b48550-bc12-4084-bd6d-eb76f2311382-host-cni-bin\") pod \"ovnkube-node-zrj9v\" (UID: \"99b48550-bc12-4084-bd6d-eb76f2311382\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v"
Sep 09 16:08:46.134862 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.134097    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/99b48550-bc12-4084-bd6d-eb76f2311382-ovn-node-metrics-cert\") pod \"ovnkube-node-zrj9v\" (UID: \"99b48550-bc12-4084-bd6d-eb76f2311382\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v"
Sep 09 16:08:46.134862 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.134123    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73b8ff58-aa37-4fab-9632-9b0f5d409902-metrics-certs\") pod \"network-metrics-daemon-t29nl\" (UID: \"73b8ff58-aa37-4fab-9632-9b0f5d409902\") " pod="openshift-multus/network-metrics-daemon-t29nl"
Sep 09 16:08:46.134862 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.134128    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/99b48550-bc12-4084-bd6d-eb76f2311382-host-cni-netd\") pod \"ovnkube-node-zrj9v\" (UID: \"99b48550-bc12-4084-bd6d-eb76f2311382\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v"
Sep 09 16:08:46.134862 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.134152    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/17c51357-18e5-4f81-bb5b-9c73e38e55bc-tmp\") pod \"tuned-rbvgk\" (UID: \"17c51357-18e5-4f81-bb5b-9c73e38e55bc\") " pod="openshift-cluster-node-tuning-operator/tuned-rbvgk"
Sep 09 16:08:46.134862 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.134178    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/17c51357-18e5-4f81-bb5b-9c73e38e55bc-etc-sysctl-d\") pod \"tuned-rbvgk\" (UID: \"17c51357-18e5-4f81-bb5b-9c73e38e55bc\") " pod="openshift-cluster-node-tuning-operator/tuned-rbvgk"
Sep 09 16:08:46.134862 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.134200    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/36d2c95e-5f46-413b-a6f5-4f0c4bd57b05-cni-binary-copy\") pod \"multus-q2pgn\" (UID: \"36d2c95e-5f46-413b-a6f5-4f0c4bd57b05\") " pod="openshift-multus/multus-q2pgn"
Sep 09 16:08:46.134862 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:46.134211    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 16:08:46.134862 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.134223    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/36d2c95e-5f46-413b-a6f5-4f0c4bd57b05-multus-socket-dir-parent\") pod \"multus-q2pgn\" (UID: \"36d2c95e-5f46-413b-a6f5-4f0c4bd57b05\") " pod="openshift-multus/multus-q2pgn"
Sep 09 16:08:46.134862 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.134243    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/99b48550-bc12-4084-bd6d-eb76f2311382-ovnkube-config\") pod \"ovnkube-node-zrj9v\" (UID: \"99b48550-bc12-4084-bd6d-eb76f2311382\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v"
Sep 09 16:08:46.134862 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:46.134276    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73b8ff58-aa37-4fab-9632-9b0f5d409902-metrics-certs podName:73b8ff58-aa37-4fab-9632-9b0f5d409902 nodeName:}" failed. No retries permitted until 2026-09-09 16:08:46.634255632 +0000 UTC m=+2.052105630 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/73b8ff58-aa37-4fab-9632-9b0f5d409902-metrics-certs") pod "network-metrics-daemon-t29nl" (UID: "73b8ff58-aa37-4fab-9632-9b0f5d409902") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 16:08:46.134862 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.134291    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/99b48550-bc12-4084-bd6d-eb76f2311382-systemd-units\") pod \"ovnkube-node-zrj9v\" (UID: \"99b48550-bc12-4084-bd6d-eb76f2311382\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v"
Sep 09 16:08:46.134862 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.134316    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/99b48550-bc12-4084-bd6d-eb76f2311382-host-slash\") pod \"ovnkube-node-zrj9v\" (UID: \"99b48550-bc12-4084-bd6d-eb76f2311382\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v"
Sep 09 16:08:46.134862 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.134342    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/36d2c95e-5f46-413b-a6f5-4f0c4bd57b05-host-var-lib-cni-multus\") pod \"multus-q2pgn\" (UID: \"36d2c95e-5f46-413b-a6f5-4f0c4bd57b05\") " pod="openshift-multus/multus-q2pgn"
Sep 09 16:08:46.135751 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.134401    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/36d2c95e-5f46-413b-a6f5-4f0c4bd57b05-multus-conf-dir\") pod \"multus-q2pgn\" (UID: \"36d2c95e-5f46-413b-a6f5-4f0c4bd57b05\") " pod="openshift-multus/multus-q2pgn"
Sep 09 16:08:46.135751 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.134427    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/36d2c95e-5f46-413b-a6f5-4f0c4bd57b05-etc-kubernetes\") pod \"multus-q2pgn\" (UID: \"36d2c95e-5f46-413b-a6f5-4f0c4bd57b05\") " pod="openshift-multus/multus-q2pgn"
Sep 09 16:08:46.135751 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.134469    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/99b48550-bc12-4084-bd6d-eb76f2311382-host-run-ovn-kubernetes\") pod \"ovnkube-node-zrj9v\" (UID: \"99b48550-bc12-4084-bd6d-eb76f2311382\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v"
Sep 09 16:08:46.135751 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.134479    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/15f41781-3a63-422e-9355-e37c98cc7c07-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-l82fl\" (UID: \"15f41781-3a63-422e-9355-e37c98cc7c07\") " pod="openshift-multus/multus-additional-cni-plugins-l82fl"
Sep 09 16:08:46.135751 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.134496    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/15f41781-3a63-422e-9355-e37c98cc7c07-tuning-conf-dir\") pod \"multus-additional-cni-plugins-l82fl\" (UID: \"15f41781-3a63-422e-9355-e37c98cc7c07\") " pod="openshift-multus/multus-additional-cni-plugins-l82fl"
Sep 09 16:08:46.135751 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.134519    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/36d2c95e-5f46-413b-a6f5-4f0c4bd57b05-hostroot\") pod \"multus-q2pgn\" (UID: \"36d2c95e-5f46-413b-a6f5-4f0c4bd57b05\") " pod="openshift-multus/multus-q2pgn"
Sep 09 16:08:46.135751 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.134558    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/36d2c95e-5f46-413b-a6f5-4f0c4bd57b05-host-var-lib-cni-multus\") pod \"multus-q2pgn\" (UID: \"36d2c95e-5f46-413b-a6f5-4f0c4bd57b05\") " pod="openshift-multus/multus-q2pgn"
Sep 09 16:08:46.135751 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.134562    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/36d2c95e-5f46-413b-a6f5-4f0c4bd57b05-multus-daemon-config\") pod \"multus-q2pgn\" (UID: \"36d2c95e-5f46-413b-a6f5-4f0c4bd57b05\") " pod="openshift-multus/multus-q2pgn"
Sep 09 16:08:46.135751 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.134580    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/99b48550-bc12-4084-bd6d-eb76f2311382-host-run-ovn-kubernetes\") pod \"ovnkube-node-zrj9v\" (UID: \"99b48550-bc12-4084-bd6d-eb76f2311382\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v"
Sep 09 16:08:46.135751 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.134596    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/99b48550-bc12-4084-bd6d-eb76f2311382-etc-openvswitch\") pod \"ovnkube-node-zrj9v\" (UID: \"99b48550-bc12-4084-bd6d-eb76f2311382\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v"
Sep 09 16:08:46.135751 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.134622    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f996fce2-faf6-474a-997d-8a9419e788d8-host\") pod \"node-ca-c7kvc\" (UID: \"f996fce2-faf6-474a-997d-8a9419e788d8\") " pod="openshift-image-registry/node-ca-c7kvc"
Sep 09 16:08:46.135751 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.134628    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/36d2c95e-5f46-413b-a6f5-4f0c4bd57b05-multus-conf-dir\") pod \"multus-q2pgn\" (UID: \"36d2c95e-5f46-413b-a6f5-4f0c4bd57b05\") " pod="openshift-multus/multus-q2pgn"
Sep 09 16:08:46.135751 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.134646    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/43ae6ccf-db13-471c-a336-733bbea3e930-host-slash\") pod \"iptables-alerter-jl8xm\" (UID: \"43ae6ccf-db13-471c-a336-733bbea3e930\") " pod="openshift-network-operator/iptables-alerter-jl8xm"
Sep 09 16:08:46.135751 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.134669    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/17c51357-18e5-4f81-bb5b-9c73e38e55bc-host\") pod \"tuned-rbvgk\" (UID: \"17c51357-18e5-4f81-bb5b-9c73e38e55bc\") " pod="openshift-cluster-node-tuning-operator/tuned-rbvgk"
Sep 09 16:08:46.135751 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.134695    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/36d2c95e-5f46-413b-a6f5-4f0c4bd57b05-host-run-k8s-cni-cncf-io\") pod \"multus-q2pgn\" (UID: \"36d2c95e-5f46-413b-a6f5-4f0c4bd57b05\") " pod="openshift-multus/multus-q2pgn"
Sep 09 16:08:46.135751 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.134720    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/36d2c95e-5f46-413b-a6f5-4f0c4bd57b05-host-var-lib-cni-bin\") pod \"multus-q2pgn\" (UID: \"36d2c95e-5f46-413b-a6f5-4f0c4bd57b05\") " pod="openshift-multus/multus-q2pgn"
Sep 09 16:08:46.135751 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.134710    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/15f41781-3a63-422e-9355-e37c98cc7c07-tuning-conf-dir\") pod \"multus-additional-cni-plugins-l82fl\" (UID: \"15f41781-3a63-422e-9355-e37c98cc7c07\") " pod="openshift-multus/multus-additional-cni-plugins-l82fl"
Sep 09 16:08:46.136533 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.134743    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/28f47047-e391-44cb-836e-df79db9e4057-dbus\") pod \"global-pull-secret-syncer-wk4dc\" (UID: \"28f47047-e391-44cb-836e-df79db9e4057\") " pod="kube-system/global-pull-secret-syncer-wk4dc"
Sep 09 16:08:46.136533 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.134780    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/15f41781-3a63-422e-9355-e37c98cc7c07-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-l82fl\" (UID: \"15f41781-3a63-422e-9355-e37c98cc7c07\") " pod="openshift-multus/multus-additional-cni-plugins-l82fl"
Sep 09 16:08:46.136533 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.134808    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7rrh\" (UniqueName: \"kubernetes.io/projected/99b48550-bc12-4084-bd6d-eb76f2311382-kube-api-access-h7rrh\") pod \"ovnkube-node-zrj9v\" (UID: \"99b48550-bc12-4084-bd6d-eb76f2311382\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v"
Sep 09 16:08:46.136533 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.134818    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/36d2c95e-5f46-413b-a6f5-4f0c4bd57b05-hostroot\") pod \"multus-q2pgn\" (UID: \"36d2c95e-5f46-413b-a6f5-4f0c4bd57b05\") " pod="openshift-multus/multus-q2pgn"
Sep 09 16:08:46.136533 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.134832    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f996fce2-faf6-474a-997d-8a9419e788d8-serviceca\") pod \"node-ca-c7kvc\" (UID: \"f996fce2-faf6-474a-997d-8a9419e788d8\") " pod="openshift-image-registry/node-ca-c7kvc"
Sep 09 16:08:46.136533 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.134524    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/43ae6ccf-db13-471c-a336-733bbea3e930-iptables-alerter-script\") pod \"iptables-alerter-jl8xm\" (UID: \"43ae6ccf-db13-471c-a336-733bbea3e930\") " pod="openshift-network-operator/iptables-alerter-jl8xm"
Sep 09 16:08:46.136533 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.134844    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/99b48550-bc12-4084-bd6d-eb76f2311382-ovnkube-config\") pod \"ovnkube-node-zrj9v\" (UID: \"99b48550-bc12-4084-bd6d-eb76f2311382\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v"
Sep 09 16:08:46.136533 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.134857    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/36d2c95e-5f46-413b-a6f5-4f0c4bd57b05-multus-cni-dir\") pod \"multus-q2pgn\" (UID: \"36d2c95e-5f46-413b-a6f5-4f0c4bd57b05\") " pod="openshift-multus/multus-q2pgn"
Sep 09 16:08:46.136533 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.134860    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/28f47047-e391-44cb-836e-df79db9e4057-dbus\") pod \"global-pull-secret-syncer-wk4dc\" (UID: \"28f47047-e391-44cb-836e-df79db9e4057\") " pod="kube-system/global-pull-secret-syncer-wk4dc"
Sep 09 16:08:46.136533 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.134883    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qq4nj\" (UniqueName: \"kubernetes.io/projected/f996fce2-faf6-474a-997d-8a9419e788d8-kube-api-access-qq4nj\") pod \"node-ca-c7kvc\" (UID: \"f996fce2-faf6-474a-997d-8a9419e788d8\") " pod="openshift-image-registry/node-ca-c7kvc"
Sep 09 16:08:46.136533 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.134894    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/43ae6ccf-db13-471c-a336-733bbea3e930-host-slash\") pod \"iptables-alerter-jl8xm\" (UID: \"43ae6ccf-db13-471c-a336-733bbea3e930\") " pod="openshift-network-operator/iptables-alerter-jl8xm"
Sep 09 16:08:46.136533 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.134909    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvkr2\" (UniqueName: \"kubernetes.io/projected/73b8ff58-aa37-4fab-9632-9b0f5d409902-kube-api-access-tvkr2\") pod \"network-metrics-daemon-t29nl\" (UID: \"73b8ff58-aa37-4fab-9632-9b0f5d409902\") " pod="openshift-multus/network-metrics-daemon-t29nl"
Sep 09 16:08:46.136533 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.134924    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/99b48550-bc12-4084-bd6d-eb76f2311382-etc-openvswitch\") pod \"ovnkube-node-zrj9v\" (UID: \"99b48550-bc12-4084-bd6d-eb76f2311382\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v"
Sep 09 16:08:46.136533 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.134924    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/17c51357-18e5-4f81-bb5b-9c73e38e55bc-host\") pod \"tuned-rbvgk\" (UID: \"17c51357-18e5-4f81-bb5b-9c73e38e55bc\") " pod="openshift-cluster-node-tuning-operator/tuned-rbvgk"
Sep 09 16:08:46.136533 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.134668    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/36d2c95e-5f46-413b-a6f5-4f0c4bd57b05-etc-kubernetes\") pod \"multus-q2pgn\" (UID: \"36d2c95e-5f46-413b-a6f5-4f0c4bd57b05\") " pod="openshift-multus/multus-q2pgn"
Sep 09 16:08:46.136533 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.134957    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/83598a1c-9f92-44aa-afa9-42556f6c0127-agent-certs\") pod \"konnectivity-agent-m7xl5\" (UID: \"83598a1c-9f92-44aa-afa9-42556f6c0127\") " pod="kube-system/konnectivity-agent-m7xl5"
Sep 09 16:08:46.136533 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.134976    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f996fce2-faf6-474a-997d-8a9419e788d8-host\") pod \"node-ca-c7kvc\" (UID: \"f996fce2-faf6-474a-997d-8a9419e788d8\") " pod="openshift-image-registry/node-ca-c7kvc"
Sep 09 16:08:46.136533 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.134980    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/15f41781-3a63-422e-9355-e37c98cc7c07-cnibin\") pod \"multus-additional-cni-plugins-l82fl\" (UID: \"15f41781-3a63-422e-9355-e37c98cc7c07\") " pod="openshift-multus/multus-additional-cni-plugins-l82fl"
Sep 09 16:08:46.137412 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.134997    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/36d2c95e-5f46-413b-a6f5-4f0c4bd57b05-host-run-k8s-cni-cncf-io\") pod \"multus-q2pgn\" (UID: \"36d2c95e-5f46-413b-a6f5-4f0c4bd57b05\") " pod="openshift-multus/multus-q2pgn"
Sep 09 16:08:46.137412 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.135008    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8b9j\" (UniqueName: \"kubernetes.io/projected/15f41781-3a63-422e-9355-e37c98cc7c07-kube-api-access-s8b9j\") pod \"multus-additional-cni-plugins-l82fl\" (UID: \"15f41781-3a63-422e-9355-e37c98cc7c07\") " pod="openshift-multus/multus-additional-cni-plugins-l82fl"
Sep 09 16:08:46.137412 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.135010    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/17c51357-18e5-4f81-bb5b-9c73e38e55bc-etc-sysctl-d\") pod \"tuned-rbvgk\" (UID: \"17c51357-18e5-4f81-bb5b-9c73e38e55bc\") " pod="openshift-cluster-node-tuning-operator/tuned-rbvgk"
Sep 09 16:08:46.137412 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.135048    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/36d2c95e-5f46-413b-a6f5-4f0c4bd57b05-multus-socket-dir-parent\") pod \"multus-q2pgn\" (UID: \"36d2c95e-5f46-413b-a6f5-4f0c4bd57b05\") " pod="openshift-multus/multus-q2pgn"
Sep 09 16:08:46.137412 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.135047    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/36d2c95e-5f46-413b-a6f5-4f0c4bd57b05-host-var-lib-cni-bin\") pod \"multus-q2pgn\" (UID: \"36d2c95e-5f46-413b-a6f5-4f0c4bd57b05\") " pod="openshift-multus/multus-q2pgn"
Sep 09 16:08:46.137412 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.135074    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/36d2c95e-5f46-413b-a6f5-4f0c4bd57b05-host-var-lib-kubelet\") pod \"multus-q2pgn\" (UID: \"36d2c95e-5f46-413b-a6f5-4f0c4bd57b05\") " pod="openshift-multus/multus-q2pgn"
Sep 09 16:08:46.137412 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.135103    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v872q\" (UniqueName: \"kubernetes.io/projected/36d2c95e-5f46-413b-a6f5-4f0c4bd57b05-kube-api-access-v872q\") pod \"multus-q2pgn\" (UID: \"36d2c95e-5f46-413b-a6f5-4f0c4bd57b05\") " pod="openshift-multus/multus-q2pgn"
Sep 09 16:08:46.137412 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.135130    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/99b48550-bc12-4084-bd6d-eb76f2311382-host-run-netns\") pod \"ovnkube-node-zrj9v\" (UID: \"99b48550-bc12-4084-bd6d-eb76f2311382\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v"
Sep 09 16:08:46.137412 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.135153    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/99b48550-bc12-4084-bd6d-eb76f2311382-run-openvswitch\") pod \"ovnkube-node-zrj9v\" (UID: \"99b48550-bc12-4084-bd6d-eb76f2311382\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v"
Sep 09 16:08:46.137412 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.135177    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/99b48550-bc12-4084-bd6d-eb76f2311382-run-ovn\") pod \"ovnkube-node-zrj9v\" (UID: \"99b48550-bc12-4084-bd6d-eb76f2311382\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v"
Sep 09 16:08:46.137412 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.135209    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/99b48550-bc12-4084-bd6d-eb76f2311382-host-kubelet\") pod \"ovnkube-node-zrj9v\" (UID: \"99b48550-bc12-4084-bd6d-eb76f2311382\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v"
Sep 09 16:08:46.137412 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.135232    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/99b48550-bc12-4084-bd6d-eb76f2311382-var-lib-openvswitch\") pod \"ovnkube-node-zrj9v\" (UID: \"99b48550-bc12-4084-bd6d-eb76f2311382\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v"
Sep 09 16:08:46.137412 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.135258    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/99b48550-bc12-4084-bd6d-eb76f2311382-log-socket\") pod \"ovnkube-node-zrj9v\" (UID: \"99b48550-bc12-4084-bd6d-eb76f2311382\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v"
Sep 09 16:08:46.137412 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.135265    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/15f41781-3a63-422e-9355-e37c98cc7c07-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-l82fl\" (UID: \"15f41781-3a63-422e-9355-e37c98cc7c07\") " pod="openshift-multus/multus-additional-cni-plugins-l82fl"
Sep 09 16:08:46.137412 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.135282    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j42zn\" (UniqueName: \"kubernetes.io/projected/43ae6ccf-db13-471c-a336-733bbea3e930-kube-api-access-j42zn\") pod \"iptables-alerter-jl8xm\" (UID: \"43ae6ccf-db13-471c-a336-733bbea3e930\") " pod="openshift-network-operator/iptables-alerter-jl8xm"
Sep 09 16:08:46.137412 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.135308    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/28f47047-e391-44cb-836e-df79db9e4057-kubelet-config\") pod \"global-pull-secret-syncer-wk4dc\" (UID: \"28f47047-e391-44cb-836e-df79db9e4057\") " pod="kube-system/global-pull-secret-syncer-wk4dc"
Sep 09 16:08:46.137412 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.135332    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/28f47047-e391-44cb-836e-df79db9e4057-original-pull-secret\") pod \"global-pull-secret-syncer-wk4dc\" (UID: \"28f47047-e391-44cb-836e-df79db9e4057\") " pod="kube-system/global-pull-secret-syncer-wk4dc"
Sep 09 16:08:46.137930 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.135358    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/17c51357-18e5-4f81-bb5b-9c73e38e55bc-etc-kubernetes\") pod \"tuned-rbvgk\" (UID: \"17c51357-18e5-4f81-bb5b-9c73e38e55bc\") " pod="openshift-cluster-node-tuning-operator/tuned-rbvgk"
Sep 09 16:08:46.137930 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.135383    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/15f41781-3a63-422e-9355-e37c98cc7c07-cni-binary-copy\") pod \"multus-additional-cni-plugins-l82fl\" (UID: \"15f41781-3a63-422e-9355-e37c98cc7c07\") " pod="openshift-multus/multus-additional-cni-plugins-l82fl"
Sep 09 16:08:46.137930 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.135410    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/36d2c95e-5f46-413b-a6f5-4f0c4bd57b05-host-run-multus-certs\") pod \"multus-q2pgn\" (UID: \"36d2c95e-5f46-413b-a6f5-4f0c4bd57b05\") " pod="openshift-multus/multus-q2pgn"
Sep 09 16:08:46.137930 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.135435    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/17c51357-18e5-4f81-bb5b-9c73e38e55bc-etc-sysctl-conf\") pod \"tuned-rbvgk\" (UID: \"17c51357-18e5-4f81-bb5b-9c73e38e55bc\") " pod="openshift-cluster-node-tuning-operator/tuned-rbvgk"
Sep 09 16:08:46.137930 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.135442    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/36d2c95e-5f46-413b-a6f5-4f0c4bd57b05-multus-cni-dir\") pod \"multus-q2pgn\" (UID: \"36d2c95e-5f46-413b-a6f5-4f0c4bd57b05\") " pod="openshift-multus/multus-q2pgn"
Sep 09 16:08:46.137930 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.135475    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/36d2c95e-5f46-413b-a6f5-4f0c4bd57b05-cni-binary-copy\") pod \"multus-q2pgn\" (UID: \"36d2c95e-5f46-413b-a6f5-4f0c4bd57b05\") " pod="openshift-multus/multus-q2pgn"
Sep 09 16:08:46.137930 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.135482    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/17c51357-18e5-4f81-bb5b-9c73e38e55bc-etc-systemd\") pod \"tuned-rbvgk\" (UID: \"17c51357-18e5-4f81-bb5b-9c73e38e55bc\") " pod="openshift-cluster-node-tuning-operator/tuned-rbvgk"
Sep 09 16:08:46.137930 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.135548    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/99b48550-bc12-4084-bd6d-eb76f2311382-log-socket\") pod \"ovnkube-node-zrj9v\" (UID: \"99b48550-bc12-4084-bd6d-eb76f2311382\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v"
Sep 09 16:08:46.137930 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.135574    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/17c51357-18e5-4f81-bb5b-9c73e38e55bc-etc-systemd\") pod \"tuned-rbvgk\" (UID: \"17c51357-18e5-4f81-bb5b-9c73e38e55bc\") " pod="openshift-cluster-node-tuning-operator/tuned-rbvgk"
Sep 09 16:08:46.137930 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.135586    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/36d2c95e-5f46-413b-a6f5-4f0c4bd57b05-host-var-lib-kubelet\") pod \"multus-q2pgn\" (UID: \"36d2c95e-5f46-413b-a6f5-4f0c4bd57b05\") " pod="openshift-multus/multus-q2pgn"
Sep 09 16:08:46.137930 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.135669    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/99b48550-bc12-4084-bd6d-eb76f2311382-env-overrides\") pod \"ovnkube-node-zrj9v\" (UID: \"99b48550-bc12-4084-bd6d-eb76f2311382\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v"
Sep 09 16:08:46.137930 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.135733    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/99b48550-bc12-4084-bd6d-eb76f2311382-host-run-netns\") pod \"ovnkube-node-zrj9v\" (UID: \"99b48550-bc12-4084-bd6d-eb76f2311382\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v"
Sep 09 16:08:46.137930 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.134884    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/99b48550-bc12-4084-bd6d-eb76f2311382-systemd-units\") pod \"ovnkube-node-zrj9v\" (UID: \"99b48550-bc12-4084-bd6d-eb76f2311382\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v"
Sep 09 16:08:46.137930 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.135813    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/99b48550-bc12-4084-bd6d-eb76f2311382-var-lib-openvswitch\") pod \"ovnkube-node-zrj9v\" (UID: \"99b48550-bc12-4084-bd6d-eb76f2311382\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v"
Sep 09 16:08:46.137930 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.135824    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/99b48550-bc12-4084-bd6d-eb76f2311382-host-kubelet\") pod \"ovnkube-node-zrj9v\" (UID: \"99b48550-bc12-4084-bd6d-eb76f2311382\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v"
Sep 09 16:08:46.137930 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.135862    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/99b48550-bc12-4084-bd6d-eb76f2311382-host-slash\") pod \"ovnkube-node-zrj9v\" (UID: \"99b48550-bc12-4084-bd6d-eb76f2311382\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v"
Sep 09 16:08:46.137930 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.135891    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/36d2c95e-5f46-413b-a6f5-4f0c4bd57b05-host-run-netns\") pod \"multus-q2pgn\" (UID: \"36d2c95e-5f46-413b-a6f5-4f0c4bd57b05\") " pod="openshift-multus/multus-q2pgn"
Sep 09 16:08:46.137930 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.135915    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/99b48550-bc12-4084-bd6d-eb76f2311382-run-systemd\") pod \"ovnkube-node-zrj9v\" (UID: \"99b48550-bc12-4084-bd6d-eb76f2311382\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v"
Sep 09 16:08:46.138580 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.135938    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/99b48550-bc12-4084-bd6d-eb76f2311382-node-log\") pod \"ovnkube-node-zrj9v\" (UID: \"99b48550-bc12-4084-bd6d-eb76f2311382\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v"
Sep 09 16:08:46.138580 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.135956    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/15f41781-3a63-422e-9355-e37c98cc7c07-cnibin\") pod \"multus-additional-cni-plugins-l82fl\" (UID: \"15f41781-3a63-422e-9355-e37c98cc7c07\") " pod="openshift-multus/multus-additional-cni-plugins-l82fl"
Sep 09 16:08:46.138580 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.135965    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/83598a1c-9f92-44aa-afa9-42556f6c0127-konnectivity-ca\") pod \"konnectivity-agent-m7xl5\" (UID: \"83598a1c-9f92-44aa-afa9-42556f6c0127\") " pod="kube-system/konnectivity-agent-m7xl5"
Sep 09 16:08:46.138580 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.135996    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/36d2c95e-5f46-413b-a6f5-4f0c4bd57b05-host-run-netns\") pod \"multus-q2pgn\" (UID: \"36d2c95e-5f46-413b-a6f5-4f0c4bd57b05\") " pod="openshift-multus/multus-q2pgn"
Sep 09 16:08:46.138580 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.135771    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/99b48550-bc12-4084-bd6d-eb76f2311382-run-openvswitch\") pod \"ovnkube-node-zrj9v\" (UID: \"99b48550-bc12-4084-bd6d-eb76f2311382\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v"
Sep 09 16:08:46.138580 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.136044    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/99b48550-bc12-4084-bd6d-eb76f2311382-node-log\") pod \"ovnkube-node-zrj9v\" (UID: \"99b48550-bc12-4084-bd6d-eb76f2311382\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v"
Sep 09 16:08:46.138580 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.136046    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/99b48550-bc12-4084-bd6d-eb76f2311382-run-systemd\") pod \"ovnkube-node-zrj9v\" (UID: \"99b48550-bc12-4084-bd6d-eb76f2311382\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v"
Sep 09 16:08:46.138580 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.136076    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/99b48550-bc12-4084-bd6d-eb76f2311382-run-ovn\") pod \"ovnkube-node-zrj9v\" (UID: \"99b48550-bc12-4084-bd6d-eb76f2311382\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v"
Sep 09 16:08:46.138580 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.136009    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/36d2c95e-5f46-413b-a6f5-4f0c4bd57b05-system-cni-dir\") pod \"multus-q2pgn\" (UID: \"36d2c95e-5f46-413b-a6f5-4f0c4bd57b05\") " pod="openshift-multus/multus-q2pgn"
Sep 09 16:08:46.138580 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.136119    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/36d2c95e-5f46-413b-a6f5-4f0c4bd57b05-cnibin\") pod \"multus-q2pgn\" (UID: \"36d2c95e-5f46-413b-a6f5-4f0c4bd57b05\") " pod="openshift-multus/multus-q2pgn"
Sep 09 16:08:46.138580 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.136142    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/99b48550-bc12-4084-bd6d-eb76f2311382-ovnkube-script-lib\") pod \"ovnkube-node-zrj9v\" (UID: \"99b48550-bc12-4084-bd6d-eb76f2311382\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v"
Sep 09 16:08:46.138580 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.136634    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/28f47047-e391-44cb-836e-df79db9e4057-kubelet-config\") pod \"global-pull-secret-syncer-wk4dc\" (UID: \"28f47047-e391-44cb-836e-df79db9e4057\") " pod="kube-system/global-pull-secret-syncer-wk4dc"
Sep 09 16:08:46.138580 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.136678    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/99b48550-bc12-4084-bd6d-eb76f2311382-ovnkube-script-lib\") pod \"ovnkube-node-zrj9v\" (UID: \"99b48550-bc12-4084-bd6d-eb76f2311382\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v"
Sep 09 16:08:46.138580 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.136696    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/83598a1c-9f92-44aa-afa9-42556f6c0127-konnectivity-ca\") pod \"konnectivity-agent-m7xl5\" (UID: \"83598a1c-9f92-44aa-afa9-42556f6c0127\") " pod="kube-system/konnectivity-agent-m7xl5"
Sep 09 16:08:46.138580 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:46.136741    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 16:08:46.138580 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.136751    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/36d2c95e-5f46-413b-a6f5-4f0c4bd57b05-system-cni-dir\") pod \"multus-q2pgn\" (UID: \"36d2c95e-5f46-413b-a6f5-4f0c4bd57b05\") " pod="openshift-multus/multus-q2pgn"
Sep 09 16:08:46.138580 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.136753    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/36d2c95e-5f46-413b-a6f5-4f0c4bd57b05-cnibin\") pod \"multus-q2pgn\" (UID: \"36d2c95e-5f46-413b-a6f5-4f0c4bd57b05\") " pod="openshift-multus/multus-q2pgn"
Sep 09 16:08:46.138580 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:46.136817    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28f47047-e391-44cb-836e-df79db9e4057-original-pull-secret podName:28f47047-e391-44cb-836e-df79db9e4057 nodeName:}" failed. No retries permitted until 2026-09-09 16:08:46.636799671 +0000 UTC m=+2.054649666 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/28f47047-e391-44cb-836e-df79db9e4057-original-pull-secret") pod "global-pull-secret-syncer-wk4dc" (UID: "28f47047-e391-44cb-836e-df79db9e4057") : object "kube-system"/"original-pull-secret" not registered
Sep 09 16:08:46.139220 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.136814    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/17c51357-18e5-4f81-bb5b-9c73e38e55bc-etc-kubernetes\") pod \"tuned-rbvgk\" (UID: \"17c51357-18e5-4f81-bb5b-9c73e38e55bc\") " pod="openshift-cluster-node-tuning-operator/tuned-rbvgk"
Sep 09 16:08:46.139220 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.136881    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/36d2c95e-5f46-413b-a6f5-4f0c4bd57b05-host-run-multus-certs\") pod \"multus-q2pgn\" (UID: \"36d2c95e-5f46-413b-a6f5-4f0c4bd57b05\") " pod="openshift-multus/multus-q2pgn"
Sep 09 16:08:46.139220 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.137069    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/36d2c95e-5f46-413b-a6f5-4f0c4bd57b05-multus-daemon-config\") pod \"multus-q2pgn\" (UID: \"36d2c95e-5f46-413b-a6f5-4f0c4bd57b05\") " pod="openshift-multus/multus-q2pgn"
Sep 09 16:08:46.139220 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.137222    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/17c51357-18e5-4f81-bb5b-9c73e38e55bc-etc-tuned\") pod \"tuned-rbvgk\" (UID: \"17c51357-18e5-4f81-bb5b-9c73e38e55bc\") " pod="openshift-cluster-node-tuning-operator/tuned-rbvgk"
Sep 09 16:08:46.139220 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.137233    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/15f41781-3a63-422e-9355-e37c98cc7c07-cni-binary-copy\") pod \"multus-additional-cni-plugins-l82fl\" (UID: \"15f41781-3a63-422e-9355-e37c98cc7c07\") " pod="openshift-multus/multus-additional-cni-plugins-l82fl"
Sep 09 16:08:46.139220 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.137280    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/17c51357-18e5-4f81-bb5b-9c73e38e55bc-etc-sysctl-conf\") pod \"tuned-rbvgk\" (UID: \"17c51357-18e5-4f81-bb5b-9c73e38e55bc\") " pod="openshift-cluster-node-tuning-operator/tuned-rbvgk"
Sep 09 16:08:46.139220 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.137319    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f996fce2-faf6-474a-997d-8a9419e788d8-serviceca\") pod \"node-ca-c7kvc\" (UID: \"f996fce2-faf6-474a-997d-8a9419e788d8\") " pod="openshift-image-registry/node-ca-c7kvc"
Sep 09 16:08:46.139220 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.138053    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/99b48550-bc12-4084-bd6d-eb76f2311382-ovn-node-metrics-cert\") pod \"ovnkube-node-zrj9v\" (UID: \"99b48550-bc12-4084-bd6d-eb76f2311382\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v"
Sep 09 16:08:46.139220 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.138303    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/83598a1c-9f92-44aa-afa9-42556f6c0127-agent-certs\") pod \"konnectivity-agent-m7xl5\" (UID: \"83598a1c-9f92-44aa-afa9-42556f6c0127\") " pod="kube-system/konnectivity-agent-m7xl5"
Sep 09 16:08:46.139220 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.139213    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/17c51357-18e5-4f81-bb5b-9c73e38e55bc-tmp\") pod \"tuned-rbvgk\" (UID: \"17c51357-18e5-4f81-bb5b-9c73e38e55bc\") " pod="openshift-cluster-node-tuning-operator/tuned-rbvgk"
Sep 09 16:08:46.139696 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:46.139371    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 16:08:46.139696 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:46.139387    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 16:08:46.139696 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:46.139399    2567 projected.go:196] Error preparing data for projected volume kube-api-access-mhw55 for pod openshift-network-diagnostics/network-check-target-p4sh4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 16:08:46.139696 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:46.139481    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3ef88440-a7bd-4a04-8293-b8121e14655e-kube-api-access-mhw55 podName:3ef88440-a7bd-4a04-8293-b8121e14655e nodeName:}" failed. No retries permitted until 2026-09-09 16:08:46.639445325 +0000 UTC m=+2.057295322 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-mhw55" (UniqueName: "kubernetes.io/projected/3ef88440-a7bd-4a04-8293-b8121e14655e-kube-api-access-mhw55") pod "network-check-target-p4sh4" (UID: "3ef88440-a7bd-4a04-8293-b8121e14655e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 16:08:46.142188 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.142161    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hbkc\" (UniqueName: \"kubernetes.io/projected/17c51357-18e5-4f81-bb5b-9c73e38e55bc-kube-api-access-6hbkc\") pod \"tuned-rbvgk\" (UID: \"17c51357-18e5-4f81-bb5b-9c73e38e55bc\") " pod="openshift-cluster-node-tuning-operator/tuned-rbvgk"
Sep 09 16:08:46.142756 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.142730    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7rrh\" (UniqueName: \"kubernetes.io/projected/99b48550-bc12-4084-bd6d-eb76f2311382-kube-api-access-h7rrh\") pod \"ovnkube-node-zrj9v\" (UID: \"99b48550-bc12-4084-bd6d-eb76f2311382\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v"
Sep 09 16:08:46.143481 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.143437    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qq4nj\" (UniqueName: \"kubernetes.io/projected/f996fce2-faf6-474a-997d-8a9419e788d8-kube-api-access-qq4nj\") pod \"node-ca-c7kvc\" (UID: \"f996fce2-faf6-474a-997d-8a9419e788d8\") " pod="openshift-image-registry/node-ca-c7kvc"
Sep 09 16:08:46.144054 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.143953    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v872q\" (UniqueName: \"kubernetes.io/projected/36d2c95e-5f46-413b-a6f5-4f0c4bd57b05-kube-api-access-v872q\") pod \"multus-q2pgn\" (UID: \"36d2c95e-5f46-413b-a6f5-4f0c4bd57b05\") " pod="openshift-multus/multus-q2pgn"
Sep 09 16:08:46.144165 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.144009    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvkr2\" (UniqueName: \"kubernetes.io/projected/73b8ff58-aa37-4fab-9632-9b0f5d409902-kube-api-access-tvkr2\") pod \"network-metrics-daemon-t29nl\" (UID: \"73b8ff58-aa37-4fab-9632-9b0f5d409902\") " pod="openshift-multus/network-metrics-daemon-t29nl"
Sep 09 16:08:46.144859 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.144837    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j42zn\" (UniqueName: \"kubernetes.io/projected/43ae6ccf-db13-471c-a336-733bbea3e930-kube-api-access-j42zn\") pod \"iptables-alerter-jl8xm\" (UID: \"43ae6ccf-db13-471c-a336-733bbea3e930\") " pod="openshift-network-operator/iptables-alerter-jl8xm"
Sep 09 16:08:46.145305 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.145287    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8b9j\" (UniqueName: \"kubernetes.io/projected/15f41781-3a63-422e-9355-e37c98cc7c07-kube-api-access-s8b9j\") pod \"multus-additional-cni-plugins-l82fl\" (UID: \"15f41781-3a63-422e-9355-e37c98cc7c07\") " pod="openshift-multus/multus-additional-cni-plugins-l82fl"
Sep 09 16:08:46.169997 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.169971    2567 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 16:08:46.241051 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:46.240982    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod271bfa508befdbc16e7141fd83859452.slice/crio-b357d8e28100a6290e354358fc1ee9224d608970ebe302f66f9abc555c6c6b56 WatchSource:0}: Error finding container b357d8e28100a6290e354358fc1ee9224d608970ebe302f66f9abc555c6c6b56: Status 404 returned error can't find the container with id b357d8e28100a6290e354358fc1ee9224d608970ebe302f66f9abc555c6c6b56
Sep 09 16:08:46.245756 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.245737    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 16:08:46.252280 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.252261    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8hqn"
Sep 09 16:08:46.263080 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:46.263050    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21b4fd71_6273_4029_889d_aa66470c26c1.slice/crio-d902e4024481c60cb14cc6c67980107412e9d27bd09e95c7133f10c9886515e9 WatchSource:0}: Error finding container d902e4024481c60cb14cc6c67980107412e9d27bd09e95c7133f10c9886515e9: Status 404 returned error can't find the container with id d902e4024481c60cb14cc6c67980107412e9d27bd09e95c7133f10c9886515e9
Sep 09 16:08:46.269349 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.269331    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-cnvzz"
Sep 09 16:08:46.277357 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:46.277333    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddeb44589_97aa_464f_95b0_61362f120948.slice/crio-df4ad8c6986479ef260600a7889496293e9888df98b3cfbe72e3316ed86d7f90 WatchSource:0}: Error finding container df4ad8c6986479ef260600a7889496293e9888df98b3cfbe72e3316ed86d7f90: Status 404 returned error can't find the container with id df4ad8c6986479ef260600a7889496293e9888df98b3cfbe72e3316ed86d7f90
Sep 09 16:08:46.278907 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.278890    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v"
Sep 09 16:08:46.287031 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:46.287009    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99b48550_bc12_4084_bd6d_eb76f2311382.slice/crio-3fc7507cdf6d0c600939d0895527897c8e7f17bb51fa0d1226a59aa915085a27 WatchSource:0}: Error finding container 3fc7507cdf6d0c600939d0895527897c8e7f17bb51fa0d1226a59aa915085a27: Status 404 returned error can't find the container with id 3fc7507cdf6d0c600939d0895527897c8e7f17bb51fa0d1226a59aa915085a27
Sep 09 16:08:46.303361 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.303338    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-m7xl5"
Sep 09 16:08:46.312025 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:46.312002    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83598a1c_9f92_44aa_afa9_42556f6c0127.slice/crio-5f2b48be19000541ff8d7131fdea9c301ea2309206366a7082b6d54e218a2a35 WatchSource:0}: Error finding container 5f2b48be19000541ff8d7131fdea9c301ea2309206366a7082b6d54e218a2a35: Status 404 returned error can't find the container with id 5f2b48be19000541ff8d7131fdea9c301ea2309206366a7082b6d54e218a2a35
Sep 09 16:08:46.323678 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.323661    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rbvgk"
Sep 09 16:08:46.332735 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:46.332710    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17c51357_18e5_4f81_bb5b_9c73e38e55bc.slice/crio-05a4d51284b96450336f9b45042989a8c9e751e9dd0ece629c14166b8d1ca2dc WatchSource:0}: Error finding container 05a4d51284b96450336f9b45042989a8c9e751e9dd0ece629c14166b8d1ca2dc: Status 404 returned error can't find the container with id 05a4d51284b96450336f9b45042989a8c9e751e9dd0ece629c14166b8d1ca2dc
Sep 09 16:08:46.339551 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.339514    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-c7kvc"
Sep 09 16:08:46.347783 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:46.347761    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf996fce2_faf6_474a_997d_8a9419e788d8.slice/crio-b414e8d7fe505b98f799ebfaf7fca3a2eac064de360ac669a496db79195bdbd4 WatchSource:0}: Error finding container b414e8d7fe505b98f799ebfaf7fca3a2eac064de360ac669a496db79195bdbd4: Status 404 returned error can't find the container with id b414e8d7fe505b98f799ebfaf7fca3a2eac064de360ac669a496db79195bdbd4
Sep 09 16:08:46.357440 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.357416    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-l82fl"
Sep 09 16:08:46.365814 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:46.365789    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15f41781_3a63_422e_9355_e37c98cc7c07.slice/crio-2bb79e223f85dc6d528d4304e406d9b6e9dc1db5dbf6749d13b7eb2614bde0aa WatchSource:0}: Error finding container 2bb79e223f85dc6d528d4304e406d9b6e9dc1db5dbf6749d13b7eb2614bde0aa: Status 404 returned error can't find the container with id 2bb79e223f85dc6d528d4304e406d9b6e9dc1db5dbf6749d13b7eb2614bde0aa
Sep 09 16:08:46.377080 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.377060    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-q2pgn"
Sep 09 16:08:46.382151 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.382136    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-jl8xm"
Sep 09 16:08:46.385532 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:46.385511    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36d2c95e_5f46_413b_a6f5_4f0c4bd57b05.slice/crio-fc801c8c3bb4a34e0723b5f39303294dde1f1263b21a04005ed258fe631f9661 WatchSource:0}: Error finding container fc801c8c3bb4a34e0723b5f39303294dde1f1263b21a04005ed258fe631f9661: Status 404 returned error can't find the container with id fc801c8c3bb4a34e0723b5f39303294dde1f1263b21a04005ed258fe631f9661
Sep 09 16:08:46.390934 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:08:46.390914    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43ae6ccf_db13_471c_a336_733bbea3e930.slice/crio-bcfaea152ead4fc0fae58ff193a82401e03b664dc1a1c1addd5b0fff495e2847 WatchSource:0}: Error finding container bcfaea152ead4fc0fae58ff193a82401e03b664dc1a1c1addd5b0fff495e2847: Status 404 returned error can't find the container with id bcfaea152ead4fc0fae58ff193a82401e03b664dc1a1c1addd5b0fff495e2847
Sep 09 16:08:46.641198 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.641093    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/28f47047-e391-44cb-836e-df79db9e4057-original-pull-secret\") pod \"global-pull-secret-syncer-wk4dc\" (UID: \"28f47047-e391-44cb-836e-df79db9e4057\") " pod="kube-system/global-pull-secret-syncer-wk4dc"
Sep 09 16:08:46.641198 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.641157    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhw55\" (UniqueName: \"kubernetes.io/projected/3ef88440-a7bd-4a04-8293-b8121e14655e-kube-api-access-mhw55\") pod \"network-check-target-p4sh4\" (UID: \"3ef88440-a7bd-4a04-8293-b8121e14655e\") " pod="openshift-network-diagnostics/network-check-target-p4sh4"
Sep 09 16:08:46.641198 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.641196    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73b8ff58-aa37-4fab-9632-9b0f5d409902-metrics-certs\") pod \"network-metrics-daemon-t29nl\" (UID: \"73b8ff58-aa37-4fab-9632-9b0f5d409902\") " pod="openshift-multus/network-metrics-daemon-t29nl"
Sep 09 16:08:46.641455 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:46.641264    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 16:08:46.641455 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:46.641298    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 16:08:46.641455 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:46.641340    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28f47047-e391-44cb-836e-df79db9e4057-original-pull-secret podName:28f47047-e391-44cb-836e-df79db9e4057 nodeName:}" failed. No retries permitted until 2026-09-09 16:08:47.64132112 +0000 UTC m=+3.059171116 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/28f47047-e391-44cb-836e-df79db9e4057-original-pull-secret") pod "global-pull-secret-syncer-wk4dc" (UID: "28f47047-e391-44cb-836e-df79db9e4057") : object "kube-system"/"original-pull-secret" not registered
Sep 09 16:08:46.641455 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:46.641357    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73b8ff58-aa37-4fab-9632-9b0f5d409902-metrics-certs podName:73b8ff58-aa37-4fab-9632-9b0f5d409902 nodeName:}" failed. No retries permitted until 2026-09-09 16:08:47.641350172 +0000 UTC m=+3.059200151 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/73b8ff58-aa37-4fab-9632-9b0f5d409902-metrics-certs") pod "network-metrics-daemon-t29nl" (UID: "73b8ff58-aa37-4fab-9632-9b0f5d409902") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 16:08:46.641455 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:46.641356    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 16:08:46.641455 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:46.641385    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 16:08:46.641455 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:46.641399    2567 projected.go:196] Error preparing data for projected volume kube-api-access-mhw55 for pod openshift-network-diagnostics/network-check-target-p4sh4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 16:08:46.641455 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:46.641449    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3ef88440-a7bd-4a04-8293-b8121e14655e-kube-api-access-mhw55 podName:3ef88440-a7bd-4a04-8293-b8121e14655e nodeName:}" failed. No retries permitted until 2026-09-09 16:08:47.64143706 +0000 UTC m=+3.059287039 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-mhw55" (UniqueName: "kubernetes.io/projected/3ef88440-a7bd-4a04-8293-b8121e14655e-kube-api-access-mhw55") pod "network-check-target-p4sh4" (UID: "3ef88440-a7bd-4a04-8293-b8121e14655e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 16:08:46.668253 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.668226    2567 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 16:08:46.969250 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.969144    2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-08 16:03:46 +0000 UTC" deadline="2028-06-09 12:51:57.812276862 +0000 UTC"
Sep 09 16:08:46.969250 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:46.969197    2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15332h43m10.84308506s"
Sep 09 16:08:47.030709 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:47.030674    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t29nl"
Sep 09 16:08:47.030898 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:47.030848    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t29nl" podUID="73b8ff58-aa37-4fab-9632-9b0f5d409902"
Sep 09 16:08:47.030978 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:47.030899    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wk4dc"
Sep 09 16:08:47.031058 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:47.031033    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wk4dc" podUID="28f47047-e391-44cb-836e-df79db9e4057"
Sep 09 16:08:47.036183 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:47.036126    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-jl8xm" event={"ID":"43ae6ccf-db13-471c-a336-733bbea3e930","Type":"ContainerStarted","Data":"bcfaea152ead4fc0fae58ff193a82401e03b664dc1a1c1addd5b0fff495e2847"}
Sep 09 16:08:47.039659 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:47.039466    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-q2pgn" event={"ID":"36d2c95e-5f46-413b-a6f5-4f0c4bd57b05","Type":"ContainerStarted","Data":"fc801c8c3bb4a34e0723b5f39303294dde1f1263b21a04005ed258fe631f9661"}
Sep 09 16:08:47.042916 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:47.042848    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rbvgk" event={"ID":"17c51357-18e5-4f81-bb5b-9c73e38e55bc","Type":"ContainerStarted","Data":"05a4d51284b96450336f9b45042989a8c9e751e9dd0ece629c14166b8d1ca2dc"}
Sep 09 16:08:47.044476 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:47.044446    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-m7xl5" event={"ID":"83598a1c-9f92-44aa-afa9-42556f6c0127","Type":"ContainerStarted","Data":"5f2b48be19000541ff8d7131fdea9c301ea2309206366a7082b6d54e218a2a35"}
Sep 09 16:08:47.046606 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:47.046480    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v" event={"ID":"99b48550-bc12-4084-bd6d-eb76f2311382","Type":"ContainerStarted","Data":"3fc7507cdf6d0c600939d0895527897c8e7f17bb51fa0d1226a59aa915085a27"}
Sep 09 16:08:47.048605 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:47.048528    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8hqn" event={"ID":"21b4fd71-6273-4029-889d-aa66470c26c1","Type":"ContainerStarted","Data":"d902e4024481c60cb14cc6c67980107412e9d27bd09e95c7133f10c9886515e9"}
Sep 09 16:08:47.050698 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:47.050638    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l82fl" event={"ID":"15f41781-3a63-422e-9355-e37c98cc7c07","Type":"ContainerStarted","Data":"2bb79e223f85dc6d528d4304e406d9b6e9dc1db5dbf6749d13b7eb2614bde0aa"}
Sep 09 16:08:47.053949 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:47.053921    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-c7kvc" event={"ID":"f996fce2-faf6-474a-997d-8a9419e788d8","Type":"ContainerStarted","Data":"b414e8d7fe505b98f799ebfaf7fca3a2eac064de360ac669a496db79195bdbd4"}
Sep 09 16:08:47.056278 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:47.056210    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-cnvzz" event={"ID":"deb44589-97aa-464f-95b0-61362f120948","Type":"ContainerStarted","Data":"df4ad8c6986479ef260600a7889496293e9888df98b3cfbe72e3316ed86d7f90"}
Sep 09 16:08:47.058388 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:47.058361    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-81.ec2.internal" event={"ID":"271bfa508befdbc16e7141fd83859452","Type":"ContainerStarted","Data":"b357d8e28100a6290e354358fc1ee9224d608970ebe302f66f9abc555c6c6b56"}
Sep 09 16:08:47.648831 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:47.648236    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73b8ff58-aa37-4fab-9632-9b0f5d409902-metrics-certs\") pod \"network-metrics-daemon-t29nl\" (UID: \"73b8ff58-aa37-4fab-9632-9b0f5d409902\") " pod="openshift-multus/network-metrics-daemon-t29nl"
Sep 09 16:08:47.648831 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:47.648326    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/28f47047-e391-44cb-836e-df79db9e4057-original-pull-secret\") pod \"global-pull-secret-syncer-wk4dc\" (UID: \"28f47047-e391-44cb-836e-df79db9e4057\") " pod="kube-system/global-pull-secret-syncer-wk4dc"
Sep 09 16:08:47.648831 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:47.648366    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhw55\" (UniqueName: \"kubernetes.io/projected/3ef88440-a7bd-4a04-8293-b8121e14655e-kube-api-access-mhw55\") pod \"network-check-target-p4sh4\" (UID: \"3ef88440-a7bd-4a04-8293-b8121e14655e\") " pod="openshift-network-diagnostics/network-check-target-p4sh4"
Sep 09 16:08:47.648831 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:47.648374    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 16:08:47.648831 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:47.648471    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73b8ff58-aa37-4fab-9632-9b0f5d409902-metrics-certs podName:73b8ff58-aa37-4fab-9632-9b0f5d409902 nodeName:}" failed. No retries permitted until 2026-09-09 16:08:49.648431857 +0000 UTC m=+5.066281839 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/73b8ff58-aa37-4fab-9632-9b0f5d409902-metrics-certs") pod "network-metrics-daemon-t29nl" (UID: "73b8ff58-aa37-4fab-9632-9b0f5d409902") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 16:08:47.648831 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:47.648523    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 16:08:47.648831 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:47.648543    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 16:08:47.648831 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:47.648557    2567 projected.go:196] Error preparing data for projected volume kube-api-access-mhw55 for pod openshift-network-diagnostics/network-check-target-p4sh4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 16:08:47.648831 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:47.648608    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3ef88440-a7bd-4a04-8293-b8121e14655e-kube-api-access-mhw55 podName:3ef88440-a7bd-4a04-8293-b8121e14655e nodeName:}" failed. No retries permitted until 2026-09-09 16:08:49.648592963 +0000 UTC m=+5.066442960 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-mhw55" (UniqueName: "kubernetes.io/projected/3ef88440-a7bd-4a04-8293-b8121e14655e-kube-api-access-mhw55") pod "network-check-target-p4sh4" (UID: "3ef88440-a7bd-4a04-8293-b8121e14655e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 16:08:47.648831 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:47.648674    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 16:08:47.648831 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:47.648708    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28f47047-e391-44cb-836e-df79db9e4057-original-pull-secret podName:28f47047-e391-44cb-836e-df79db9e4057 nodeName:}" failed. No retries permitted until 2026-09-09 16:08:49.648697301 +0000 UTC m=+5.066547285 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/28f47047-e391-44cb-836e-df79db9e4057-original-pull-secret") pod "global-pull-secret-syncer-wk4dc" (UID: "28f47047-e391-44cb-836e-df79db9e4057") : object "kube-system"/"original-pull-secret" not registered
Sep 09 16:08:47.970086 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:47.969961    2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-08 16:03:46 +0000 UTC" deadline="2028-02-22 16:17:36.568607383 +0000 UTC"
Sep 09 16:08:47.970086 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:47.970022    2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12744h8m48.598589759s"
Sep 09 16:08:48.030285 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:48.030252    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p4sh4"
Sep 09 16:08:48.030491 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:48.030388    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p4sh4" podUID="3ef88440-a7bd-4a04-8293-b8121e14655e"
Sep 09 16:08:48.067320 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:48.067274    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-81.ec2.internal" event={"ID":"fd9da504dd49d348bc16b790735150bc","Type":"ContainerStarted","Data":"9a3bbc925cdc7df591b4ac9c883dd1be8bb153881b6fc74a21373ef7eb971d47"}
Sep 09 16:08:49.030507 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:49.030407    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wk4dc"
Sep 09 16:08:49.031066 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:49.030407    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t29nl"
Sep 09 16:08:49.031066 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:49.030590    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wk4dc" podUID="28f47047-e391-44cb-836e-df79db9e4057"
Sep 09 16:08:49.031066 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:49.030903    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t29nl" podUID="73b8ff58-aa37-4fab-9632-9b0f5d409902"
Sep 09 16:08:49.668579 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:49.667679    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/28f47047-e391-44cb-836e-df79db9e4057-original-pull-secret\") pod \"global-pull-secret-syncer-wk4dc\" (UID: \"28f47047-e391-44cb-836e-df79db9e4057\") " pod="kube-system/global-pull-secret-syncer-wk4dc"
Sep 09 16:08:49.668579 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:49.667743    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhw55\" (UniqueName: \"kubernetes.io/projected/3ef88440-a7bd-4a04-8293-b8121e14655e-kube-api-access-mhw55\") pod \"network-check-target-p4sh4\" (UID: \"3ef88440-a7bd-4a04-8293-b8121e14655e\") " pod="openshift-network-diagnostics/network-check-target-p4sh4"
Sep 09 16:08:49.668579 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:49.667786    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73b8ff58-aa37-4fab-9632-9b0f5d409902-metrics-certs\") pod \"network-metrics-daemon-t29nl\" (UID: \"73b8ff58-aa37-4fab-9632-9b0f5d409902\") " pod="openshift-multus/network-metrics-daemon-t29nl"
Sep 09 16:08:49.668579 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:49.667914    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 16:08:49.668579 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:49.667977    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73b8ff58-aa37-4fab-9632-9b0f5d409902-metrics-certs podName:73b8ff58-aa37-4fab-9632-9b0f5d409902 nodeName:}" failed. No retries permitted until 2026-09-09 16:08:53.667958778 +0000 UTC m=+9.085808760 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/73b8ff58-aa37-4fab-9632-9b0f5d409902-metrics-certs") pod "network-metrics-daemon-t29nl" (UID: "73b8ff58-aa37-4fab-9632-9b0f5d409902") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 16:08:49.668579 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:49.668347    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 16:08:49.668579 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:49.668394    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28f47047-e391-44cb-836e-df79db9e4057-original-pull-secret podName:28f47047-e391-44cb-836e-df79db9e4057 nodeName:}" failed. No retries permitted until 2026-09-09 16:08:53.668379584 +0000 UTC m=+9.086229565 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/28f47047-e391-44cb-836e-df79db9e4057-original-pull-secret") pod "global-pull-secret-syncer-wk4dc" (UID: "28f47047-e391-44cb-836e-df79db9e4057") : object "kube-system"/"original-pull-secret" not registered
Sep 09 16:08:49.668579 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:49.668490    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 16:08:49.668579 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:49.668510    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 16:08:49.668579 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:49.668519    2567 projected.go:196] Error preparing data for projected volume kube-api-access-mhw55 for pod openshift-network-diagnostics/network-check-target-p4sh4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 16:08:49.668579 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:49.668545    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3ef88440-a7bd-4a04-8293-b8121e14655e-kube-api-access-mhw55 podName:3ef88440-a7bd-4a04-8293-b8121e14655e nodeName:}" failed. No retries permitted until 2026-09-09 16:08:53.668537746 +0000 UTC m=+9.086387722 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-mhw55" (UniqueName: "kubernetes.io/projected/3ef88440-a7bd-4a04-8293-b8121e14655e-kube-api-access-mhw55") pod "network-check-target-p4sh4" (UID: "3ef88440-a7bd-4a04-8293-b8121e14655e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 16:08:50.029850 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:50.029769    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p4sh4"
Sep 09 16:08:50.030007 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:50.029891    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p4sh4" podUID="3ef88440-a7bd-4a04-8293-b8121e14655e"
Sep 09 16:08:51.031832 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:51.031795    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t29nl"
Sep 09 16:08:51.032262 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:51.031949    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t29nl" podUID="73b8ff58-aa37-4fab-9632-9b0f5d409902"
Sep 09 16:08:51.032262 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:51.032005    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wk4dc"
Sep 09 16:08:51.032262 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:51.032123    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wk4dc" podUID="28f47047-e391-44cb-836e-df79db9e4057"
Sep 09 16:08:52.030308 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:52.029843    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p4sh4"
Sep 09 16:08:52.030308 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:52.029976    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p4sh4" podUID="3ef88440-a7bd-4a04-8293-b8121e14655e"
Sep 09 16:08:53.030046 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:53.029961    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t29nl"
Sep 09 16:08:53.030530 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:53.030105    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t29nl" podUID="73b8ff58-aa37-4fab-9632-9b0f5d409902"
Sep 09 16:08:53.030530 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:53.030148    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wk4dc"
Sep 09 16:08:53.030530 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:53.030245    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wk4dc" podUID="28f47047-e391-44cb-836e-df79db9e4057"
Sep 09 16:08:53.703527 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:53.703224    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/28f47047-e391-44cb-836e-df79db9e4057-original-pull-secret\") pod \"global-pull-secret-syncer-wk4dc\" (UID: \"28f47047-e391-44cb-836e-df79db9e4057\") " pod="kube-system/global-pull-secret-syncer-wk4dc"
Sep 09 16:08:53.703527 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:53.703266    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhw55\" (UniqueName: \"kubernetes.io/projected/3ef88440-a7bd-4a04-8293-b8121e14655e-kube-api-access-mhw55\") pod \"network-check-target-p4sh4\" (UID: \"3ef88440-a7bd-4a04-8293-b8121e14655e\") " pod="openshift-network-diagnostics/network-check-target-p4sh4"
Sep 09 16:08:53.703527 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:53.703296    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73b8ff58-aa37-4fab-9632-9b0f5d409902-metrics-certs\") pod \"network-metrics-daemon-t29nl\" (UID: \"73b8ff58-aa37-4fab-9632-9b0f5d409902\") " pod="openshift-multus/network-metrics-daemon-t29nl"
Sep 09 16:08:53.703527 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:53.703394    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 16:08:53.703527 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:53.703408    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 16:08:53.703527 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:53.703410    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 16:08:53.703527 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:53.703434    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 16:08:53.703527 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:53.703445    2567 projected.go:196] Error preparing data for projected volume kube-api-access-mhw55 for pod openshift-network-diagnostics/network-check-target-p4sh4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 16:08:53.703527 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:53.703476    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73b8ff58-aa37-4fab-9632-9b0f5d409902-metrics-certs podName:73b8ff58-aa37-4fab-9632-9b0f5d409902 nodeName:}" failed. No retries permitted until 2026-09-09 16:09:01.703445861 +0000 UTC m=+17.121295851 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/73b8ff58-aa37-4fab-9632-9b0f5d409902-metrics-certs") pod "network-metrics-daemon-t29nl" (UID: "73b8ff58-aa37-4fab-9632-9b0f5d409902") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 16:08:53.703527 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:53.703494    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28f47047-e391-44cb-836e-df79db9e4057-original-pull-secret podName:28f47047-e391-44cb-836e-df79db9e4057 nodeName:}" failed. No retries permitted until 2026-09-09 16:09:01.703484686 +0000 UTC m=+17.121334663 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/28f47047-e391-44cb-836e-df79db9e4057-original-pull-secret") pod "global-pull-secret-syncer-wk4dc" (UID: "28f47047-e391-44cb-836e-df79db9e4057") : object "kube-system"/"original-pull-secret" not registered
Sep 09 16:08:53.703527 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:53.703507    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3ef88440-a7bd-4a04-8293-b8121e14655e-kube-api-access-mhw55 podName:3ef88440-a7bd-4a04-8293-b8121e14655e nodeName:}" failed. No retries permitted until 2026-09-09 16:09:01.703499527 +0000 UTC m=+17.121349507 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-mhw55" (UniqueName: "kubernetes.io/projected/3ef88440-a7bd-4a04-8293-b8121e14655e-kube-api-access-mhw55") pod "network-check-target-p4sh4" (UID: "3ef88440-a7bd-4a04-8293-b8121e14655e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 16:08:54.029633 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:54.029542    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p4sh4"
Sep 09 16:08:54.029810 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:54.029679    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p4sh4" podUID="3ef88440-a7bd-4a04-8293-b8121e14655e"
Sep 09 16:08:55.031093 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:55.031060    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t29nl"
Sep 09 16:08:55.031567 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:55.031170    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t29nl" podUID="73b8ff58-aa37-4fab-9632-9b0f5d409902"
Sep 09 16:08:55.031667 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:55.031645    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wk4dc"
Sep 09 16:08:55.031796 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:55.031747    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wk4dc" podUID="28f47047-e391-44cb-836e-df79db9e4057"
Sep 09 16:08:56.029788 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:56.029763    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p4sh4"
Sep 09 16:08:56.029916 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:56.029868    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p4sh4" podUID="3ef88440-a7bd-4a04-8293-b8121e14655e"
Sep 09 16:08:57.032578 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:57.032541    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t29nl"
Sep 09 16:08:57.033045 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:57.032690    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t29nl" podUID="73b8ff58-aa37-4fab-9632-9b0f5d409902"
Sep 09 16:08:57.033110 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:57.033057    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wk4dc"
Sep 09 16:08:57.033166 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:57.033151    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wk4dc" podUID="28f47047-e391-44cb-836e-df79db9e4057"
Sep 09 16:08:58.029665 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:58.029628    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p4sh4"
Sep 09 16:08:58.029855 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:58.029761    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p4sh4" podUID="3ef88440-a7bd-4a04-8293-b8121e14655e"
Sep 09 16:08:59.030045 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:59.030008    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t29nl"
Sep 09 16:08:59.030045 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:08:59.030054    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wk4dc"
Sep 09 16:08:59.030614 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:59.030152    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t29nl" podUID="73b8ff58-aa37-4fab-9632-9b0f5d409902"
Sep 09 16:08:59.030614 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:08:59.030303    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wk4dc" podUID="28f47047-e391-44cb-836e-df79db9e4057"
Sep 09 16:09:00.030054 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:00.030016    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p4sh4"
Sep 09 16:09:00.030539 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:09:00.030144    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p4sh4" podUID="3ef88440-a7bd-4a04-8293-b8121e14655e"
Sep 09 16:09:01.030286 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:01.030251    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t29nl"
Sep 09 16:09:01.030685 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:09:01.030371    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t29nl" podUID="73b8ff58-aa37-4fab-9632-9b0f5d409902"
Sep 09 16:09:01.030685 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:01.030438    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wk4dc"
Sep 09 16:09:01.030685 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:09:01.030554    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wk4dc" podUID="28f47047-e391-44cb-836e-df79db9e4057"
Sep 09 16:09:01.766764 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:01.766726    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/28f47047-e391-44cb-836e-df79db9e4057-original-pull-secret\") pod \"global-pull-secret-syncer-wk4dc\" (UID: \"28f47047-e391-44cb-836e-df79db9e4057\") " pod="kube-system/global-pull-secret-syncer-wk4dc"
Sep 09 16:09:01.766976 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:01.766781    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhw55\" (UniqueName: \"kubernetes.io/projected/3ef88440-a7bd-4a04-8293-b8121e14655e-kube-api-access-mhw55\") pod \"network-check-target-p4sh4\" (UID: \"3ef88440-a7bd-4a04-8293-b8121e14655e\") " pod="openshift-network-diagnostics/network-check-target-p4sh4"
Sep 09 16:09:01.766976 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:01.766817    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73b8ff58-aa37-4fab-9632-9b0f5d409902-metrics-certs\") pod \"network-metrics-daemon-t29nl\" (UID: \"73b8ff58-aa37-4fab-9632-9b0f5d409902\") " pod="openshift-multus/network-metrics-daemon-t29nl"
Sep 09 16:09:01.766976 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:09:01.766911    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 16:09:01.766976 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:09:01.766924    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 16:09:01.767154 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:09:01.766973    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 16:09:01.767154 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:09:01.766983    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73b8ff58-aa37-4fab-9632-9b0f5d409902-metrics-certs podName:73b8ff58-aa37-4fab-9632-9b0f5d409902 nodeName:}" failed. No retries permitted until 2026-09-09 16:09:17.766963408 +0000 UTC m=+33.184813403 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/73b8ff58-aa37-4fab-9632-9b0f5d409902-metrics-certs") pod "network-metrics-daemon-t29nl" (UID: "73b8ff58-aa37-4fab-9632-9b0f5d409902") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 16:09:01.767154 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:09:01.766998    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 16:09:01.767154 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:09:01.767001    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28f47047-e391-44cb-836e-df79db9e4057-original-pull-secret podName:28f47047-e391-44cb-836e-df79db9e4057 nodeName:}" failed. No retries permitted until 2026-09-09 16:09:17.766992846 +0000 UTC m=+33.184842829 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/28f47047-e391-44cb-836e-df79db9e4057-original-pull-secret") pod "global-pull-secret-syncer-wk4dc" (UID: "28f47047-e391-44cb-836e-df79db9e4057") : object "kube-system"/"original-pull-secret" not registered
Sep 09 16:09:01.767154 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:09:01.767012    2567 projected.go:196] Error preparing data for projected volume kube-api-access-mhw55 for pod openshift-network-diagnostics/network-check-target-p4sh4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 16:09:01.767154 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:09:01.767059    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3ef88440-a7bd-4a04-8293-b8121e14655e-kube-api-access-mhw55 podName:3ef88440-a7bd-4a04-8293-b8121e14655e nodeName:}" failed. No retries permitted until 2026-09-09 16:09:17.767042117 +0000 UTC m=+33.184892097 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-mhw55" (UniqueName: "kubernetes.io/projected/3ef88440-a7bd-4a04-8293-b8121e14655e-kube-api-access-mhw55") pod "network-check-target-p4sh4" (UID: "3ef88440-a7bd-4a04-8293-b8121e14655e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 16:09:02.029578 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:02.029484    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p4sh4"
Sep 09 16:09:02.029750 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:09:02.029631    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p4sh4" podUID="3ef88440-a7bd-4a04-8293-b8121e14655e"
Sep 09 16:09:03.030124 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:03.030094    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wk4dc"
Sep 09 16:09:03.030637 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:03.030094    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t29nl"
Sep 09 16:09:03.030637 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:09:03.030225    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wk4dc" podUID="28f47047-e391-44cb-836e-df79db9e4057"
Sep 09 16:09:03.030637 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:09:03.030289    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t29nl" podUID="73b8ff58-aa37-4fab-9632-9b0f5d409902"
Sep 09 16:09:04.029710 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:04.029674    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p4sh4"
Sep 09 16:09:04.029907 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:09:04.029794    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p4sh4" podUID="3ef88440-a7bd-4a04-8293-b8121e14655e"
Sep 09 16:09:05.030944 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:05.030705    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t29nl"
Sep 09 16:09:05.031312 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:05.030757    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wk4dc"
Sep 09 16:09:05.031312 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:09:05.031058    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t29nl" podUID="73b8ff58-aa37-4fab-9632-9b0f5d409902"
Sep 09 16:09:05.031312 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:09:05.031159    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wk4dc" podUID="28f47047-e391-44cb-836e-df79db9e4057"
Sep 09 16:09:05.101959 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:05.101924    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rbvgk" event={"ID":"17c51357-18e5-4f81-bb5b-9c73e38e55bc","Type":"ContainerStarted","Data":"c023843df2c4a0603db7ca7316f01efec5b6d638dd65d1fb15163cb06dc75813"}
Sep 09 16:09:05.105201 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:05.105172    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-81.ec2.internal" event={"ID":"fd9da504dd49d348bc16b790735150bc","Type":"ContainerStarted","Data":"6a001b5ced29f784d1c81dc4646c04a2ab8cae8ba78000b8fb678aa13d9a3cdb"}
Sep 09 16:09:05.129111 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:05.129062    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-81.ec2.internal" podStartSLOduration=19.129048539 podStartE2EDuration="19.129048539s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 16:08:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 16:09:05.12873856 +0000 UTC m=+20.546588559" watchObservedRunningTime="2026-09-09 16:09:05.129048539 +0000 UTC m=+20.546898538"
Sep 09 16:09:05.129246 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:05.129156    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-rbvgk" podStartSLOduration=1.619716558 podStartE2EDuration="20.129152623s" totalImagesPullingTime="18.509436065s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 16:08:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 16:09:05.11696102 +0000 UTC m=+20.534811019" watchObservedRunningTime="2026-09-09 16:09:05.129152623 +0000 UTC m=+20.547002683"
Sep 09 16:09:06.030449 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:06.030167    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p4sh4"
Sep 09 16:09:06.030581 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:09:06.030562    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p4sh4" podUID="3ef88440-a7bd-4a04-8293-b8121e14655e"
Sep 09 16:09:06.110944 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:06.110921    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-m7xl5" event={"ID":"83598a1c-9f92-44aa-afa9-42556f6c0127","Type":"ContainerStarted","Data":"c6007b0eab1561e3169b13d9d27829d11e2c1c0b15db40944cc408f84caa0ed4"}
Sep 09 16:09:06.113497 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:06.113386    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v" event={"ID":"99b48550-bc12-4084-bd6d-eb76f2311382","Type":"ContainerStarted","Data":"f34189b7b5b2033cb5f7b76fe15d1240abae9e1f1fbe3f006409cb88a8052b28"}
Sep 09 16:09:06.113497 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:06.113441    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v" event={"ID":"99b48550-bc12-4084-bd6d-eb76f2311382","Type":"ContainerStarted","Data":"b8e083bf44d12e08754f5f6dfe016458581df950768dccdd0cb8f7ec305cc7a4"}
Sep 09 16:09:06.113596 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:06.113502    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v" event={"ID":"99b48550-bc12-4084-bd6d-eb76f2311382","Type":"ContainerStarted","Data":"cb54175583283eeea4a0362c87c39a06ff3ec8ddfc540e7786cc6b475e6e2fd2"}
Sep 09 16:09:06.113596 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:06.113544    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v" event={"ID":"99b48550-bc12-4084-bd6d-eb76f2311382","Type":"ContainerStarted","Data":"128d6b3d3e7e18bf727b98d419f7c87c91d07d6d02846b910f3107917c4d6434"}
Sep 09 16:09:06.114945 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:06.114924    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8hqn" event={"ID":"21b4fd71-6273-4029-889d-aa66470c26c1","Type":"ContainerStarted","Data":"44c8c0dd54c2827b925e42bc66fc9bd08440c2aba6cbb14edb0d6f6b62c67084"}
Sep 09 16:09:06.116390 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:06.116367    2567 generic.go:356] "Generic (PLEG): container finished" podID="15f41781-3a63-422e-9355-e37c98cc7c07" containerID="eeaeda0a64418fe4f200c47937742c6b85e4e1df2e0adf4ed286c3693376ae92" exitCode=0
Sep 09 16:09:06.116508 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:06.116434    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l82fl" event={"ID":"15f41781-3a63-422e-9355-e37c98cc7c07","Type":"ContainerDied","Data":"eeaeda0a64418fe4f200c47937742c6b85e4e1df2e0adf4ed286c3693376ae92"}
Sep 09 16:09:06.117953 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:06.117836    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-c7kvc" event={"ID":"f996fce2-faf6-474a-997d-8a9419e788d8","Type":"ContainerStarted","Data":"472fff0ce2932f10384ed2f9bc5bf44adb4b483e98cc473513a8c33d8544e582"}
Sep 09 16:09:06.119511 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:06.119326    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-cnvzz" event={"ID":"deb44589-97aa-464f-95b0-61362f120948","Type":"ContainerStarted","Data":"69df6a052d1d9a1a0fe1885362079f80e779c36d30aa3128beb2e61530544130"}
Sep 09 16:09:06.121088 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:06.121064    2567 generic.go:356] "Generic (PLEG): container finished" podID="271bfa508befdbc16e7141fd83859452" containerID="7f1dd8ff018d631661558c4b1363674f956517c36486fd309f5fc4ffe4a74e30" exitCode=0
Sep 09 16:09:06.121182 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:06.121128    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-81.ec2.internal" event={"ID":"271bfa508befdbc16e7141fd83859452","Type":"ContainerDied","Data":"7f1dd8ff018d631661558c4b1363674f956517c36486fd309f5fc4ffe4a74e30"}
Sep 09 16:09:06.122725 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:06.122699    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-q2pgn" event={"ID":"36d2c95e-5f46-413b-a6f5-4f0c4bd57b05","Type":"ContainerStarted","Data":"f629dfab4ae6c1f3f63778c5c7493da3197331a3fee8a06a75600d91d2fedfdb"}
Sep 09 16:09:06.139862 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:06.139821    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-m7xl5" podStartSLOduration=10.193089912 podStartE2EDuration="21.139806249s" totalImagesPullingTime="10.946716337s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 16:08:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 16:09:06.123969681 +0000 UTC m=+21.541819680" watchObservedRunningTime="2026-09-09 16:09:06.139806249 +0000 UTC m=+21.557656229"
Sep 09 16:09:06.154801 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:06.154666    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-cnvzz" podStartSLOduration=11.413742011 podStartE2EDuration="21.15464918s" totalImagesPullingTime="9.740907169s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 16:08:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 16:09:06.154577469 +0000 UTC m=+21.572427469" watchObservedRunningTime="2026-09-09 16:09:06.15464918 +0000 UTC m=+21.572499179"
Sep 09 16:09:06.168668 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:06.168618    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-c7kvc" podStartSLOduration=2.708324045 podStartE2EDuration="21.168604554s" totalImagesPullingTime="18.460280509s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 16:08:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 16:09:06.167882966 +0000 UTC m=+21.585732966" watchObservedRunningTime="2026-09-09 16:09:06.168604554 +0000 UTC m=+21.586454554"
Sep 09 16:09:06.203304 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:06.203159    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-q2pgn" podStartSLOduration=2.709072222 podStartE2EDuration="21.203144449s" totalImagesPullingTime="18.494072227s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 16:08:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 16:09:06.202218044 +0000 UTC m=+21.620068045" watchObservedRunningTime="2026-09-09 16:09:06.203144449 +0000 UTC m=+21.620994447"
Sep 09 16:09:07.030175 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:07.030139    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wk4dc"
Sep 09 16:09:07.030365 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:09:07.030272    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wk4dc" podUID="28f47047-e391-44cb-836e-df79db9e4057"
Sep 09 16:09:07.030365 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:07.030333    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t29nl"
Sep 09 16:09:07.030521 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:09:07.030497    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t29nl" podUID="73b8ff58-aa37-4fab-9632-9b0f5d409902"
Sep 09 16:09:07.126701 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:07.126661    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-jl8xm" event={"ID":"43ae6ccf-db13-471c-a336-733bbea3e930","Type":"ContainerStarted","Data":"2982ecfbf260b52ccf058ee0992b3127f1374fa3eec92505fbf0c53f2b069594"}
Sep 09 16:09:07.129634 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:07.129603    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v" event={"ID":"99b48550-bc12-4084-bd6d-eb76f2311382","Type":"ContainerStarted","Data":"8320bd86eb1323730543844eb220b70cb462cb5f2479e39db67ba187c82efc3c"}
Sep 09 16:09:07.129766 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:07.129673    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v" event={"ID":"99b48550-bc12-4084-bd6d-eb76f2311382","Type":"ContainerStarted","Data":"fcc9d73d66a2a8ec39e4eba56ad738a88d9953023d92ff57a580d0dd69c6f5aa"}
Sep 09 16:09:07.131268 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:07.131240    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-81.ec2.internal" event={"ID":"271bfa508befdbc16e7141fd83859452","Type":"ContainerStarted","Data":"5756ed8623cea1c0b3aca6ce4ccedc046886697a8855c2bb971b751f804a6e75"}
Sep 09 16:09:07.139959 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:07.139919    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-jl8xm" podStartSLOduration=11.595479247 podStartE2EDuration="22.139907694s" totalImagesPullingTime="10.544428447s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 16:08:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 16:09:07.139556503 +0000 UTC m=+22.557406501" watchObservedRunningTime="2026-09-09 16:09:07.139907694 +0000 UTC m=+22.557757693"
Sep 09 16:09:07.152279 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:07.152236    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-81.ec2.internal" podStartSLOduration=21.152221923 podStartE2EDuration="21.152221923s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 16:08:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 16:09:07.151798364 +0000 UTC m=+22.569648362" watchObservedRunningTime="2026-09-09 16:09:07.152221923 +0000 UTC m=+22.570071921"
Sep 09 16:09:07.729107 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:07.729067    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-m7xl5"
Sep 09 16:09:07.729717 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:07.729693    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-m7xl5"
Sep 09 16:09:08.030673 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:08.030592    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p4sh4"
Sep 09 16:09:08.030843 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:09:08.030733    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p4sh4" podUID="3ef88440-a7bd-4a04-8293-b8121e14655e"
Sep 09 16:09:08.545560 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:08.545535    2567 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 09 16:09:09.003303 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:09.003175    2567 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-09T16:09:08.545558064Z","UUID":"88bf738d-0883-4916-bbd2-0380640da3e0","Handler":null,"Name":"","Endpoint":""}
Sep 09 16:09:09.004628 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:09.004606    2567 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 09 16:09:09.004628 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:09.004641    2567 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 09 16:09:09.029597 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:09.029532    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t29nl"
Sep 09 16:09:09.029597 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:09.029563    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wk4dc"
Sep 09 16:09:09.029809 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:09:09.029670    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t29nl" podUID="73b8ff58-aa37-4fab-9632-9b0f5d409902"
Sep 09 16:09:09.029870 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:09:09.029825    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wk4dc" podUID="28f47047-e391-44cb-836e-df79db9e4057"
Sep 09 16:09:09.139564 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:09.139530    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v" event={"ID":"99b48550-bc12-4084-bd6d-eb76f2311382","Type":"ContainerStarted","Data":"d40b38737b0c94c68f85fd2ef8c9aec8bacb2aef5d9d844d15998bcfc27cafa3"}
Sep 09 16:09:09.141663 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:09.141635    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8hqn" event={"ID":"21b4fd71-6273-4029-889d-aa66470c26c1","Type":"ContainerStarted","Data":"ccfaf031de2b5f237a9e897d3cba4a0cb63af3992b374a11ccfe56f7dc452f88"}
Sep 09 16:09:09.141800 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:09.141643    2567 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 09 16:09:10.030526 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:10.030486    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p4sh4"
Sep 09 16:09:10.030978 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:09:10.030629    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p4sh4" podUID="3ef88440-a7bd-4a04-8293-b8121e14655e"
Sep 09 16:09:11.029856 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:11.029711    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t29nl"
Sep 09 16:09:11.029986 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:11.029775    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wk4dc"
Sep 09 16:09:11.029986 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:09:11.029945    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t29nl" podUID="73b8ff58-aa37-4fab-9632-9b0f5d409902"
Sep 09 16:09:11.030067 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:09:11.030014    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wk4dc" podUID="28f47047-e391-44cb-836e-df79db9e4057"
Sep 09 16:09:11.150414 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:11.150381    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v" event={"ID":"99b48550-bc12-4084-bd6d-eb76f2311382","Type":"ContainerStarted","Data":"2ec89208f5fed786b35ea6af9d03a4b0e9d8e5865f5336604663e9edc5021243"}
Sep 09 16:09:11.150884 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:11.150624    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v"
Sep 09 16:09:11.166405 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:11.166374    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v"
Sep 09 16:09:11.177941 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:11.177889    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v" podStartSLOduration=6.774667473 podStartE2EDuration="26.177873997s" totalImagesPullingTime="19.403206524s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 16:08:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 16:09:11.177420923 +0000 UTC m=+26.595270922" watchObservedRunningTime="2026-09-09 16:09:11.177873997 +0000 UTC m=+26.595723996"
Sep 09 16:09:12.030264 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:12.030233    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p4sh4"
Sep 09 16:09:12.030442 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:09:12.030325    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p4sh4" podUID="3ef88440-a7bd-4a04-8293-b8121e14655e"
Sep 09 16:09:12.154613 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:12.154585    2567 generic.go:356] "Generic (PLEG): container finished" podID="15f41781-3a63-422e-9355-e37c98cc7c07" containerID="021b0c21a0e47464504f30fe0f370c7a0a1fddaee236fb5a3e64c7b3faf057e4" exitCode=0
Sep 09 16:09:12.155259 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:12.154670    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l82fl" event={"ID":"15f41781-3a63-422e-9355-e37c98cc7c07","Type":"ContainerDied","Data":"021b0c21a0e47464504f30fe0f370c7a0a1fddaee236fb5a3e64c7b3faf057e4"}
Sep 09 16:09:12.155259 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:12.155210    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v"
Sep 09 16:09:12.155363 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:12.155279    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v"
Sep 09 16:09:12.172027 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:12.172005    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v"
Sep 09 16:09:12.412619 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:12.412581    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-m7xl5"
Sep 09 16:09:12.412842 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:12.412750    2567 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 09 16:09:12.413203 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:12.413169    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-m7xl5"
Sep 09 16:09:12.943051 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:12.942857    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-p4sh4"]
Sep 09 16:09:12.943207 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:12.943136    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p4sh4"
Sep 09 16:09:12.943276 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:09:12.943225    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p4sh4" podUID="3ef88440-a7bd-4a04-8293-b8121e14655e"
Sep 09 16:09:12.952285 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:12.952259    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wk4dc"]
Sep 09 16:09:12.952566 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:12.952317    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-t29nl"]
Sep 09 16:09:12.952566 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:12.952408    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t29nl"
Sep 09 16:09:12.952566 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:09:12.952523    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t29nl" podUID="73b8ff58-aa37-4fab-9632-9b0f5d409902"
Sep 09 16:09:12.952875 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:12.952857    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wk4dc"
Sep 09 16:09:12.952978 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:09:12.952958    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wk4dc" podUID="28f47047-e391-44cb-836e-df79db9e4057"
Sep 09 16:09:13.159057 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:13.159024    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8hqn" event={"ID":"21b4fd71-6273-4029-889d-aa66470c26c1","Type":"ContainerStarted","Data":"995b0dfac796641e726a101e3ecda09dfbd9e681d5a20a7b3d6580a196a49c48"}
Sep 09 16:09:13.161273 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:13.161252    2567 generic.go:356] "Generic (PLEG): container finished" podID="15f41781-3a63-422e-9355-e37c98cc7c07" containerID="5107f5694fe151f2b178a978f1bd4974520b58f1cfaadc8a703b442e57c986ae" exitCode=0
Sep 09 16:09:13.161349 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:13.161318    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l82fl" event={"ID":"15f41781-3a63-422e-9355-e37c98cc7c07","Type":"ContainerDied","Data":"5107f5694fe151f2b178a978f1bd4974520b58f1cfaadc8a703b442e57c986ae"}
Sep 09 16:09:13.175927 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:13.175877    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8hqn" podStartSLOduration=3.480952235 podStartE2EDuration="28.175860834s" totalImagesPullingTime="24.694908599s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 16:08:45 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 16:09:13.175587707 +0000 UTC m=+28.593437704" watchObservedRunningTime="2026-09-09 16:09:13.175860834 +0000 UTC m=+28.593710832"
Sep 09 16:09:15.030220 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:15.030189    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p4sh4"
Sep 09 16:09:15.030710 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:15.030250    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t29nl"
Sep 09 16:09:15.030710 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:09:15.030264    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p4sh4" podUID="3ef88440-a7bd-4a04-8293-b8121e14655e"
Sep 09 16:09:15.030710 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:09:15.030354    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t29nl" podUID="73b8ff58-aa37-4fab-9632-9b0f5d409902"
Sep 09 16:09:15.030710 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:15.030402    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wk4dc"
Sep 09 16:09:15.030710 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:09:15.030485    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wk4dc" podUID="28f47047-e391-44cb-836e-df79db9e4057"
Sep 09 16:09:15.170216 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:15.170182    2567 generic.go:356] "Generic (PLEG): container finished" podID="15f41781-3a63-422e-9355-e37c98cc7c07" containerID="c0f1abb38435c905471fb9240ea2d0b8bb55b07d3ac408847fc7d6131b7eecfa" exitCode=0
Sep 09 16:09:15.170216 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:15.170215    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l82fl" event={"ID":"15f41781-3a63-422e-9355-e37c98cc7c07","Type":"ContainerDied","Data":"c0f1abb38435c905471fb9240ea2d0b8bb55b07d3ac408847fc7d6131b7eecfa"}
Sep 09 16:09:17.030285 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:17.030255    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p4sh4"
Sep 09 16:09:17.030285 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:17.030274    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t29nl"
Sep 09 16:09:17.030706 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:17.030274    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wk4dc"
Sep 09 16:09:17.030706 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:09:17.030377    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p4sh4" podUID="3ef88440-a7bd-4a04-8293-b8121e14655e"
Sep 09 16:09:17.030706 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:09:17.030449    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t29nl" podUID="73b8ff58-aa37-4fab-9632-9b0f5d409902"
Sep 09 16:09:17.030706 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:09:17.030547    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wk4dc" podUID="28f47047-e391-44cb-836e-df79db9e4057"
Sep 09 16:09:17.779764 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:17.779724    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/28f47047-e391-44cb-836e-df79db9e4057-original-pull-secret\") pod \"global-pull-secret-syncer-wk4dc\" (UID: \"28f47047-e391-44cb-836e-df79db9e4057\") " pod="kube-system/global-pull-secret-syncer-wk4dc"
Sep 09 16:09:17.779764 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:17.779767    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhw55\" (UniqueName: \"kubernetes.io/projected/3ef88440-a7bd-4a04-8293-b8121e14655e-kube-api-access-mhw55\") pod \"network-check-target-p4sh4\" (UID: \"3ef88440-a7bd-4a04-8293-b8121e14655e\") " pod="openshift-network-diagnostics/network-check-target-p4sh4"
Sep 09 16:09:17.780029 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:17.779790    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73b8ff58-aa37-4fab-9632-9b0f5d409902-metrics-certs\") pod \"network-metrics-daemon-t29nl\" (UID: \"73b8ff58-aa37-4fab-9632-9b0f5d409902\") " pod="openshift-multus/network-metrics-daemon-t29nl"
Sep 09 16:09:17.780029 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:09:17.779876    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 16:09:17.780029 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:09:17.779881    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 16:09:17.780029 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:09:17.779903    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 16:09:17.780029 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:09:17.779927    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 16:09:17.780029 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:09:17.779942    2567 projected.go:196] Error preparing data for projected volume kube-api-access-mhw55 for pod openshift-network-diagnostics/network-check-target-p4sh4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 16:09:17.780029 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:09:17.779928    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73b8ff58-aa37-4fab-9632-9b0f5d409902-metrics-certs podName:73b8ff58-aa37-4fab-9632-9b0f5d409902 nodeName:}" failed. No retries permitted until 2026-09-09 16:09:49.779914193 +0000 UTC m=+65.197764171 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/73b8ff58-aa37-4fab-9632-9b0f5d409902-metrics-certs") pod "network-metrics-daemon-t29nl" (UID: "73b8ff58-aa37-4fab-9632-9b0f5d409902") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 16:09:17.780029 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:09:17.779998    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28f47047-e391-44cb-836e-df79db9e4057-original-pull-secret podName:28f47047-e391-44cb-836e-df79db9e4057 nodeName:}" failed. No retries permitted until 2026-09-09 16:09:49.779979187 +0000 UTC m=+65.197829176 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/28f47047-e391-44cb-836e-df79db9e4057-original-pull-secret") pod "global-pull-secret-syncer-wk4dc" (UID: "28f47047-e391-44cb-836e-df79db9e4057") : object "kube-system"/"original-pull-secret" not registered
Sep 09 16:09:17.780029 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:09:17.780021    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3ef88440-a7bd-4a04-8293-b8121e14655e-kube-api-access-mhw55 podName:3ef88440-a7bd-4a04-8293-b8121e14655e nodeName:}" failed. No retries permitted until 2026-09-09 16:09:49.780012717 +0000 UTC m=+65.197862708 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-mhw55" (UniqueName: "kubernetes.io/projected/3ef88440-a7bd-4a04-8293-b8121e14655e-kube-api-access-mhw55") pod "network-check-target-p4sh4" (UID: "3ef88440-a7bd-4a04-8293-b8121e14655e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 16:09:19.030223 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:19.030186    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p4sh4"
Sep 09 16:09:19.030777 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:19.030193    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wk4dc"
Sep 09 16:09:19.030777 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:09:19.030316    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p4sh4" podUID="3ef88440-a7bd-4a04-8293-b8121e14655e"
Sep 09 16:09:19.030777 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:09:19.030398    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wk4dc" podUID="28f47047-e391-44cb-836e-df79db9e4057"
Sep 09 16:09:19.030777 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:19.030196    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t29nl"
Sep 09 16:09:19.030777 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:09:19.030536    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t29nl" podUID="73b8ff58-aa37-4fab-9632-9b0f5d409902"
Sep 09 16:09:19.333116 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:19.333086    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-81.ec2.internal" event="NodeReady"
Sep 09 16:09:19.333282 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:19.333209    2567 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 09 16:09:19.382475 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:19.382431    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-frbps"]
Sep 09 16:09:19.404759 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:19.404733    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-tbdg5"]
Sep 09 16:09:19.404925 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:19.404869    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-frbps"
Sep 09 16:09:19.407727 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:19.407370    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 09 16:09:19.407727 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:19.407420    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-pgn4p\""
Sep 09 16:09:19.407727 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:19.407378    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 09 16:09:19.428824 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:19.428796    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-frbps"]
Sep 09 16:09:19.428972 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:19.428836    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tbdg5"]
Sep 09 16:09:19.428972 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:19.428920    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tbdg5"
Sep 09 16:09:19.432592 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:19.432570    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 09 16:09:19.436199 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:19.436181    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 09 16:09:19.437735 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:19.437714    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 09 16:09:19.437911 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:19.437894    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-74c5r\""
Sep 09 16:09:19.490926 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:19.490894    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8f8cbff3-9dfd-4406-9c22-260bd366d946-config-volume\") pod \"dns-default-frbps\" (UID: \"8f8cbff3-9dfd-4406-9c22-260bd366d946\") " pod="openshift-dns/dns-default-frbps"
Sep 09 16:09:19.491105 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:19.490936    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8f8cbff3-9dfd-4406-9c22-260bd366d946-metrics-tls\") pod \"dns-default-frbps\" (UID: \"8f8cbff3-9dfd-4406-9c22-260bd366d946\") " pod="openshift-dns/dns-default-frbps"
Sep 09 16:09:19.491105 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:19.490957    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f8cbff3-9dfd-4406-9c22-260bd366d946-tmp-dir\") pod \"dns-default-frbps\" (UID: \"8f8cbff3-9dfd-4406-9c22-260bd366d946\") " pod="openshift-dns/dns-default-frbps"
Sep 09 16:09:19.491105 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:19.490973    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szh7g\" (UniqueName: \"kubernetes.io/projected/8f8cbff3-9dfd-4406-9c22-260bd366d946-kube-api-access-szh7g\") pod \"dns-default-frbps\" (UID: \"8f8cbff3-9dfd-4406-9c22-260bd366d946\") " pod="openshift-dns/dns-default-frbps"
Sep 09 16:09:19.591642 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:19.591549    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8f8cbff3-9dfd-4406-9c22-260bd366d946-config-volume\") pod \"dns-default-frbps\" (UID: \"8f8cbff3-9dfd-4406-9c22-260bd366d946\") " pod="openshift-dns/dns-default-frbps"
Sep 09 16:09:19.591642 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:19.591611    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8f8cbff3-9dfd-4406-9c22-260bd366d946-metrics-tls\") pod \"dns-default-frbps\" (UID: \"8f8cbff3-9dfd-4406-9c22-260bd366d946\") " pod="openshift-dns/dns-default-frbps"
Sep 09 16:09:19.591642 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:19.591643    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/65ebbfa4-b996-49d0-ae3e-2a1b8c3d7677-cert\") pod \"ingress-canary-tbdg5\" (UID: \"65ebbfa4-b996-49d0-ae3e-2a1b8c3d7677\") " pod="openshift-ingress-canary/ingress-canary-tbdg5"
Sep 09 16:09:19.591913 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:19.591669    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f8cbff3-9dfd-4406-9c22-260bd366d946-tmp-dir\") pod \"dns-default-frbps\" (UID: \"8f8cbff3-9dfd-4406-9c22-260bd366d946\") " pod="openshift-dns/dns-default-frbps"
Sep 09 16:09:19.591913 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:19.591697    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szh7g\" (UniqueName: \"kubernetes.io/projected/8f8cbff3-9dfd-4406-9c22-260bd366d946-kube-api-access-szh7g\") pod \"dns-default-frbps\" (UID: \"8f8cbff3-9dfd-4406-9c22-260bd366d946\") " pod="openshift-dns/dns-default-frbps"
Sep 09 16:09:19.591913 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:09:19.591783    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 16:09:19.591913 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:19.591852    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxmln\" (UniqueName: \"kubernetes.io/projected/65ebbfa4-b996-49d0-ae3e-2a1b8c3d7677-kube-api-access-qxmln\") pod \"ingress-canary-tbdg5\" (UID: \"65ebbfa4-b996-49d0-ae3e-2a1b8c3d7677\") " pod="openshift-ingress-canary/ingress-canary-tbdg5"
Sep 09 16:09:19.591913 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:09:19.591890    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f8cbff3-9dfd-4406-9c22-260bd366d946-metrics-tls podName:8f8cbff3-9dfd-4406-9c22-260bd366d946 nodeName:}" failed. No retries permitted until 2026-09-09 16:09:20.091832701 +0000 UTC m=+35.509682679 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8f8cbff3-9dfd-4406-9c22-260bd366d946-metrics-tls") pod "dns-default-frbps" (UID: "8f8cbff3-9dfd-4406-9c22-260bd366d946") : secret "dns-default-metrics-tls" not found
Sep 09 16:09:19.592166 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:19.592064    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f8cbff3-9dfd-4406-9c22-260bd366d946-tmp-dir\") pod \"dns-default-frbps\" (UID: \"8f8cbff3-9dfd-4406-9c22-260bd366d946\") " pod="openshift-dns/dns-default-frbps"
Sep 09 16:09:19.592249 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:19.592230    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8f8cbff3-9dfd-4406-9c22-260bd366d946-config-volume\") pod \"dns-default-frbps\" (UID: \"8f8cbff3-9dfd-4406-9c22-260bd366d946\") " pod="openshift-dns/dns-default-frbps"
Sep 09 16:09:19.604554 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:19.604352    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-szh7g\" (UniqueName: \"kubernetes.io/projected/8f8cbff3-9dfd-4406-9c22-260bd366d946-kube-api-access-szh7g\") pod \"dns-default-frbps\" (UID: \"8f8cbff3-9dfd-4406-9c22-260bd366d946\") " pod="openshift-dns/dns-default-frbps"
Sep 09 16:09:19.693057 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:19.693021    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/65ebbfa4-b996-49d0-ae3e-2a1b8c3d7677-cert\") pod \"ingress-canary-tbdg5\" (UID: \"65ebbfa4-b996-49d0-ae3e-2a1b8c3d7677\") " pod="openshift-ingress-canary/ingress-canary-tbdg5"
Sep 09 16:09:19.693257 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:19.693077    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxmln\" (UniqueName: \"kubernetes.io/projected/65ebbfa4-b996-49d0-ae3e-2a1b8c3d7677-kube-api-access-qxmln\") pod \"ingress-canary-tbdg5\" (UID: \"65ebbfa4-b996-49d0-ae3e-2a1b8c3d7677\") " pod="openshift-ingress-canary/ingress-canary-tbdg5"
Sep 09 16:09:19.696202 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:19.696176    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/65ebbfa4-b996-49d0-ae3e-2a1b8c3d7677-cert\") pod \"ingress-canary-tbdg5\" (UID: \"65ebbfa4-b996-49d0-ae3e-2a1b8c3d7677\") " pod="openshift-ingress-canary/ingress-canary-tbdg5"
Sep 09 16:09:19.701977 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:19.701948    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxmln\" (UniqueName: \"kubernetes.io/projected/65ebbfa4-b996-49d0-ae3e-2a1b8c3d7677-kube-api-access-qxmln\") pod \"ingress-canary-tbdg5\" (UID: \"65ebbfa4-b996-49d0-ae3e-2a1b8c3d7677\") " pod="openshift-ingress-canary/ingress-canary-tbdg5"
Sep 09 16:09:19.738947 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:19.738912    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tbdg5"
Sep 09 16:09:20.054927 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:20.054845    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tbdg5"]
Sep 09 16:09:20.058528 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:09:20.058499    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65ebbfa4_b996_49d0_ae3e_2a1b8c3d7677.slice/crio-65830da708fdb18eaf9d49c42523d61d0e80da380263c888bae5639381578d35 WatchSource:0}: Error finding container 65830da708fdb18eaf9d49c42523d61d0e80da380263c888bae5639381578d35: Status 404 returned error can't find the container with id 65830da708fdb18eaf9d49c42523d61d0e80da380263c888bae5639381578d35
Sep 09 16:09:20.096632 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:20.096601    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8f8cbff3-9dfd-4406-9c22-260bd366d946-metrics-tls\") pod \"dns-default-frbps\" (UID: \"8f8cbff3-9dfd-4406-9c22-260bd366d946\") " pod="openshift-dns/dns-default-frbps"
Sep 09 16:09:20.096783 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:09:20.096717    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 16:09:20.096783 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:09:20.096778    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f8cbff3-9dfd-4406-9c22-260bd366d946-metrics-tls podName:8f8cbff3-9dfd-4406-9c22-260bd366d946 nodeName:}" failed. No retries permitted until 2026-09-09 16:09:21.096761576 +0000 UTC m=+36.514611554 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8f8cbff3-9dfd-4406-9c22-260bd366d946-metrics-tls") pod "dns-default-frbps" (UID: "8f8cbff3-9dfd-4406-9c22-260bd366d946") : secret "dns-default-metrics-tls" not found
Sep 09 16:09:20.182594 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:20.182558    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tbdg5" event={"ID":"65ebbfa4-b996-49d0-ae3e-2a1b8c3d7677","Type":"ContainerStarted","Data":"65830da708fdb18eaf9d49c42523d61d0e80da380263c888bae5639381578d35"}
Sep 09 16:09:21.030021 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:21.029975    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p4sh4"
Sep 09 16:09:21.030021 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:21.030019    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wk4dc"
Sep 09 16:09:21.030225 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:21.029984    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t29nl"
Sep 09 16:09:21.034082 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:21.034059    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-trsmq\""
Sep 09 16:09:21.034227 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:21.034096    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 09 16:09:21.034227 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:21.034099    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 09 16:09:21.034227 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:21.034061    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zj7bv\""
Sep 09 16:09:21.034416 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:21.034398    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 09 16:09:21.034509 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:21.034408    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 09 16:09:21.105762 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:21.105726    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8f8cbff3-9dfd-4406-9c22-260bd366d946-metrics-tls\") pod \"dns-default-frbps\" (UID: \"8f8cbff3-9dfd-4406-9c22-260bd366d946\") " pod="openshift-dns/dns-default-frbps"
Sep 09 16:09:21.106324 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:09:21.105903    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 16:09:21.106324 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:09:21.105985    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f8cbff3-9dfd-4406-9c22-260bd366d946-metrics-tls podName:8f8cbff3-9dfd-4406-9c22-260bd366d946 nodeName:}" failed. No retries permitted until 2026-09-09 16:09:23.105961149 +0000 UTC m=+38.523811128 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8f8cbff3-9dfd-4406-9c22-260bd366d946-metrics-tls") pod "dns-default-frbps" (UID: "8f8cbff3-9dfd-4406-9c22-260bd366d946") : secret "dns-default-metrics-tls" not found
Sep 09 16:09:23.119645 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:23.119400    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8f8cbff3-9dfd-4406-9c22-260bd366d946-metrics-tls\") pod \"dns-default-frbps\" (UID: \"8f8cbff3-9dfd-4406-9c22-260bd366d946\") " pod="openshift-dns/dns-default-frbps"
Sep 09 16:09:23.120199 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:09:23.119562    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 16:09:23.120199 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:09:23.119762    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f8cbff3-9dfd-4406-9c22-260bd366d946-metrics-tls podName:8f8cbff3-9dfd-4406-9c22-260bd366d946 nodeName:}" failed. No retries permitted until 2026-09-09 16:09:27.119747372 +0000 UTC m=+42.537597349 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8f8cbff3-9dfd-4406-9c22-260bd366d946-metrics-tls") pod "dns-default-frbps" (UID: "8f8cbff3-9dfd-4406-9c22-260bd366d946") : secret "dns-default-metrics-tls" not found
Sep 09 16:09:23.193541 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:23.193502    2567 generic.go:356] "Generic (PLEG): container finished" podID="15f41781-3a63-422e-9355-e37c98cc7c07" containerID="0d00c144e46a36882acffd4c24f4de4e65fd682cbfa927ada7b4160754d3b61d" exitCode=0
Sep 09 16:09:23.193716 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:23.193580    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l82fl" event={"ID":"15f41781-3a63-422e-9355-e37c98cc7c07","Type":"ContainerDied","Data":"0d00c144e46a36882acffd4c24f4de4e65fd682cbfa927ada7b4160754d3b61d"}
Sep 09 16:09:24.199937 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:24.199787    2567 generic.go:356] "Generic (PLEG): container finished" podID="15f41781-3a63-422e-9355-e37c98cc7c07" containerID="4d77e14fd3b0309fd403221fe25784859207d3298ddd661c911c8de01ae72825" exitCode=0
Sep 09 16:09:24.199937 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:24.199846    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l82fl" event={"ID":"15f41781-3a63-422e-9355-e37c98cc7c07","Type":"ContainerDied","Data":"4d77e14fd3b0309fd403221fe25784859207d3298ddd661c911c8de01ae72825"}
Sep 09 16:09:25.205873 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:25.205787    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l82fl" event={"ID":"15f41781-3a63-422e-9355-e37c98cc7c07","Type":"ContainerStarted","Data":"528f096f2073845631f786bcc701a01cfd5e4b56efcd5eca48924d41860ebfe9"}
Sep 09 16:09:25.207231 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:25.207210    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tbdg5" event={"ID":"65ebbfa4-b996-49d0-ae3e-2a1b8c3d7677","Type":"ContainerStarted","Data":"365cde2c08a101a0cf8cb456c7eed2830e1e765490be95047ec98e81121915d2"}
Sep 09 16:09:25.236301 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:25.236255    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-l82fl" podStartSLOduration=16.880752114 podStartE2EDuration="40.236240204s" totalImagesPullingTime="23.35548809s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 16:08:45 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 16:09:25.234985156 +0000 UTC m=+40.652835156" watchObservedRunningTime="2026-09-09 16:09:25.236240204 +0000 UTC m=+40.654090203"
Sep 09 16:09:25.257317 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:25.257267    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-tbdg5" podStartSLOduration=1.426604682 podStartE2EDuration="6.257254571s" totalImagesPullingTime="4.830649889s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 16:09:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 16:09:25.256043824 +0000 UTC m=+40.673893822" watchObservedRunningTime="2026-09-09 16:09:25.257254571 +0000 UTC m=+40.675104569"
Sep 09 16:09:27.148236 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:27.148198    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8f8cbff3-9dfd-4406-9c22-260bd366d946-metrics-tls\") pod \"dns-default-frbps\" (UID: \"8f8cbff3-9dfd-4406-9c22-260bd366d946\") " pod="openshift-dns/dns-default-frbps"
Sep 09 16:09:27.148669 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:09:27.148335    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 16:09:27.148669 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:09:27.148395    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f8cbff3-9dfd-4406-9c22-260bd366d946-metrics-tls podName:8f8cbff3-9dfd-4406-9c22-260bd366d946 nodeName:}" failed. No retries permitted until 2026-09-09 16:09:35.148381565 +0000 UTC m=+50.566231542 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8f8cbff3-9dfd-4406-9c22-260bd366d946-metrics-tls") pod "dns-default-frbps" (UID: "8f8cbff3-9dfd-4406-9c22-260bd366d946") : secret "dns-default-metrics-tls" not found
Sep 09 16:09:34.536052 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:34.536008    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b8f54bb66-xffhq"]
Sep 09 16:09:34.547794 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:34.547761    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b8f54bb66-xffhq"]
Sep 09 16:09:34.547923 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:34.547855    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b8f54bb66-xffhq"
Sep 09 16:09:34.550067 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:34.550044    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 09 16:09:34.550211 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:34.550089    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 09 16:09:34.550822 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:34.550802    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-g44tx\""
Sep 09 16:09:34.550984 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:34.550937    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Sep 09 16:09:34.550984 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:34.550963    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 09 16:09:34.702959 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:34.702924    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bf814534-30a8-4914-9651-3f3ceb4e8e27-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5b8f54bb66-xffhq\" (UID: \"bf814534-30a8-4914-9651-3f3ceb4e8e27\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b8f54bb66-xffhq"
Sep 09 16:09:34.703133 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:34.702996    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdx2r\" (UniqueName: \"kubernetes.io/projected/bf814534-30a8-4914-9651-3f3ceb4e8e27-kube-api-access-qdx2r\") pod \"managed-serviceaccount-addon-agent-5b8f54bb66-xffhq\" (UID: \"bf814534-30a8-4914-9651-3f3ceb4e8e27\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b8f54bb66-xffhq"
Sep 09 16:09:34.803389 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:34.803296    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bf814534-30a8-4914-9651-3f3ceb4e8e27-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5b8f54bb66-xffhq\" (UID: \"bf814534-30a8-4914-9651-3f3ceb4e8e27\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b8f54bb66-xffhq"
Sep 09 16:09:34.803389 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:34.803383    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdx2r\" (UniqueName: \"kubernetes.io/projected/bf814534-30a8-4914-9651-3f3ceb4e8e27-kube-api-access-qdx2r\") pod \"managed-serviceaccount-addon-agent-5b8f54bb66-xffhq\" (UID: \"bf814534-30a8-4914-9651-3f3ceb4e8e27\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b8f54bb66-xffhq"
Sep 09 16:09:34.810590 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:34.810569    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bf814534-30a8-4914-9651-3f3ceb4e8e27-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5b8f54bb66-xffhq\" (UID: \"bf814534-30a8-4914-9651-3f3ceb4e8e27\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b8f54bb66-xffhq"
Sep 09 16:09:34.816004 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:34.815981    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdx2r\" (UniqueName: \"kubernetes.io/projected/bf814534-30a8-4914-9651-3f3ceb4e8e27-kube-api-access-qdx2r\") pod \"managed-serviceaccount-addon-agent-5b8f54bb66-xffhq\" (UID: \"bf814534-30a8-4914-9651-3f3ceb4e8e27\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b8f54bb66-xffhq"
Sep 09 16:09:34.868281 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:34.868246    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b8f54bb66-xffhq"
Sep 09 16:09:35.156173 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:35.156142    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b8f54bb66-xffhq"]
Sep 09 16:09:35.159465 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:09:35.159426    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbf814534_30a8_4914_9651_3f3ceb4e8e27.slice/crio-05c18173048909d8ce5e1f16579da600545ca1283a49e283740baa60f4507f0d WatchSource:0}: Error finding container 05c18173048909d8ce5e1f16579da600545ca1283a49e283740baa60f4507f0d: Status 404 returned error can't find the container with id 05c18173048909d8ce5e1f16579da600545ca1283a49e283740baa60f4507f0d
Sep 09 16:09:35.206496 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:35.206447    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8f8cbff3-9dfd-4406-9c22-260bd366d946-metrics-tls\") pod \"dns-default-frbps\" (UID: \"8f8cbff3-9dfd-4406-9c22-260bd366d946\") " pod="openshift-dns/dns-default-frbps"
Sep 09 16:09:35.206639 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:09:35.206602    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 16:09:35.206680 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:09:35.206667    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f8cbff3-9dfd-4406-9c22-260bd366d946-metrics-tls podName:8f8cbff3-9dfd-4406-9c22-260bd366d946 nodeName:}" failed. No retries permitted until 2026-09-09 16:09:51.206648755 +0000 UTC m=+66.624498745 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8f8cbff3-9dfd-4406-9c22-260bd366d946-metrics-tls") pod "dns-default-frbps" (UID: "8f8cbff3-9dfd-4406-9c22-260bd366d946") : secret "dns-default-metrics-tls" not found
Sep 09 16:09:35.232306 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:35.232274    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b8f54bb66-xffhq" event={"ID":"bf814534-30a8-4914-9651-3f3ceb4e8e27","Type":"ContainerStarted","Data":"05c18173048909d8ce5e1f16579da600545ca1283a49e283740baa60f4507f0d"}
Sep 09 16:09:38.241674 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:38.241583    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b8f54bb66-xffhq" event={"ID":"bf814534-30a8-4914-9651-3f3ceb4e8e27","Type":"ContainerStarted","Data":"7215b96da9c2c10ebf6d6092fc1ee29e8b0b30542f94c889524071e6f972fbda"}
Sep 09 16:09:38.255612 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:38.255561    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b8f54bb66-xffhq" podStartSLOduration=1.5031075440000001 podStartE2EDuration="4.255546339s" totalImagesPullingTime="2.752438795s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 16:09:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 16:09:38.255193668 +0000 UTC m=+53.673043673" watchObservedRunningTime="2026-09-09 16:09:38.255546339 +0000 UTC m=+53.673396338"
Sep 09 16:09:44.177497 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:44.177447    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zrj9v"
Sep 09 16:09:49.813973 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:49.813921    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/28f47047-e391-44cb-836e-df79db9e4057-original-pull-secret\") pod \"global-pull-secret-syncer-wk4dc\" (UID: \"28f47047-e391-44cb-836e-df79db9e4057\") " pod="kube-system/global-pull-secret-syncer-wk4dc"
Sep 09 16:09:49.813973 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:49.813984    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhw55\" (UniqueName: \"kubernetes.io/projected/3ef88440-a7bd-4a04-8293-b8121e14655e-kube-api-access-mhw55\") pod \"network-check-target-p4sh4\" (UID: \"3ef88440-a7bd-4a04-8293-b8121e14655e\") " pod="openshift-network-diagnostics/network-check-target-p4sh4"
Sep 09 16:09:49.814565 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:49.814021    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73b8ff58-aa37-4fab-9632-9b0f5d409902-metrics-certs\") pod \"network-metrics-daemon-t29nl\" (UID: \"73b8ff58-aa37-4fab-9632-9b0f5d409902\") " pod="openshift-multus/network-metrics-daemon-t29nl"
Sep 09 16:09:49.816867 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:49.816845    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 09 16:09:49.816948 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:49.816846    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 09 16:09:49.816948 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:49.816899    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 09 16:09:49.825028 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:09:49.825009    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 09 16:09:49.825097 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:09:49.825075    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73b8ff58-aa37-4fab-9632-9b0f5d409902-metrics-certs podName:73b8ff58-aa37-4fab-9632-9b0f5d409902 nodeName:}" failed. No retries permitted until 2026-09-09 16:10:53.825059359 +0000 UTC m=+129.242909336 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/73b8ff58-aa37-4fab-9632-9b0f5d409902-metrics-certs") pod "network-metrics-daemon-t29nl" (UID: "73b8ff58-aa37-4fab-9632-9b0f5d409902") : secret "metrics-daemon-secret" not found
Sep 09 16:09:49.827141 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:49.827122    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 09 16:09:49.827340 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:49.827322    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/28f47047-e391-44cb-836e-df79db9e4057-original-pull-secret\") pod \"global-pull-secret-syncer-wk4dc\" (UID: \"28f47047-e391-44cb-836e-df79db9e4057\") " pod="kube-system/global-pull-secret-syncer-wk4dc"
Sep 09 16:09:49.838251 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:49.838231    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhw55\" (UniqueName: \"kubernetes.io/projected/3ef88440-a7bd-4a04-8293-b8121e14655e-kube-api-access-mhw55\") pod \"network-check-target-p4sh4\" (UID: \"3ef88440-a7bd-4a04-8293-b8121e14655e\") " pod="openshift-network-diagnostics/network-check-target-p4sh4"
Sep 09 16:09:49.844324 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:49.844305    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-trsmq\""
Sep 09 16:09:49.852087 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:49.852070    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p4sh4"
Sep 09 16:09:49.854372 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:49.854358    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wk4dc"
Sep 09 16:09:50.125126 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:50.125093    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wk4dc"]
Sep 09 16:09:50.128087 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:50.128061    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-p4sh4"]
Sep 09 16:09:50.128652 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:09:50.128633    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28f47047_e391_44cb_836e_df79db9e4057.slice/crio-ee55f94bda00ef1ded1772dd221a95cedf48cbe3898caf90f3e6bb38399f1d41 WatchSource:0}: Error finding container ee55f94bda00ef1ded1772dd221a95cedf48cbe3898caf90f3e6bb38399f1d41: Status 404 returned error can't find the container with id ee55f94bda00ef1ded1772dd221a95cedf48cbe3898caf90f3e6bb38399f1d41
Sep 09 16:09:50.130936 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:09:50.130915    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ef88440_a7bd_4a04_8293_b8121e14655e.slice/crio-de19dcd6e4c84eeaaea70dea63fc75081f216d121c7f89df9b7cb598f186bf8b WatchSource:0}: Error finding container de19dcd6e4c84eeaaea70dea63fc75081f216d121c7f89df9b7cb598f186bf8b: Status 404 returned error can't find the container with id de19dcd6e4c84eeaaea70dea63fc75081f216d121c7f89df9b7cb598f186bf8b
Sep 09 16:09:50.270595 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:50.270556    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wk4dc" event={"ID":"28f47047-e391-44cb-836e-df79db9e4057","Type":"ContainerStarted","Data":"ee55f94bda00ef1ded1772dd221a95cedf48cbe3898caf90f3e6bb38399f1d41"}
Sep 09 16:09:50.271650 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:50.271627    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-p4sh4" event={"ID":"3ef88440-a7bd-4a04-8293-b8121e14655e","Type":"ContainerStarted","Data":"de19dcd6e4c84eeaaea70dea63fc75081f216d121c7f89df9b7cb598f186bf8b"}
Sep 09 16:09:51.222807 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:51.222768    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8f8cbff3-9dfd-4406-9c22-260bd366d946-metrics-tls\") pod \"dns-default-frbps\" (UID: \"8f8cbff3-9dfd-4406-9c22-260bd366d946\") " pod="openshift-dns/dns-default-frbps"
Sep 09 16:09:51.223204 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:09:51.222919    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 16:09:51.223204 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:09:51.222986    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f8cbff3-9dfd-4406-9c22-260bd366d946-metrics-tls podName:8f8cbff3-9dfd-4406-9c22-260bd366d946 nodeName:}" failed. No retries permitted until 2026-09-09 16:10:23.222970695 +0000 UTC m=+98.640820672 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8f8cbff3-9dfd-4406-9c22-260bd366d946-metrics-tls") pod "dns-default-frbps" (UID: "8f8cbff3-9dfd-4406-9c22-260bd366d946") : secret "dns-default-metrics-tls" not found
Sep 09 16:09:57.292065 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:57.292023    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-p4sh4" event={"ID":"3ef88440-a7bd-4a04-8293-b8121e14655e","Type":"ContainerStarted","Data":"dea085d801ff33d21a758cbc084f13917dd2d7e47a4ca954a9e13fc185bf0986"}
Sep 09 16:09:57.292609 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:57.292126    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-p4sh4"
Sep 09 16:09:57.293442 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:57.293421    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wk4dc" event={"ID":"28f47047-e391-44cb-836e-df79db9e4057","Type":"ContainerStarted","Data":"b00489ead2577214336b3e5647f6d168cce07999011415103af0e808ca90521a"}
Sep 09 16:09:57.305871 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:57.305831    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-p4sh4" podStartSLOduration=66.027490885 podStartE2EDuration="1m12.305818146s" totalImagesPullingTime="6.278327261s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 16:08:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 16:09:57.305427617 +0000 UTC m=+72.723277614" watchObservedRunningTime="2026-09-09 16:09:57.305818146 +0000 UTC m=+72.723668148"
Sep 09 16:09:57.319601 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:09:57.319523    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-wk4dc" podStartSLOduration=65.666588086 podStartE2EDuration="1m12.319506211s" totalImagesPullingTime="6.652918125s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 16:08:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 16:09:57.319487566 +0000 UTC m=+72.737337563" watchObservedRunningTime="2026-09-09 16:09:57.319506211 +0000 UTC m=+72.737356210"
Sep 09 16:10:23.239101 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:10:23.239050    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8f8cbff3-9dfd-4406-9c22-260bd366d946-metrics-tls\") pod \"dns-default-frbps\" (UID: \"8f8cbff3-9dfd-4406-9c22-260bd366d946\") " pod="openshift-dns/dns-default-frbps"
Sep 09 16:10:23.239752 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:10:23.239196    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 16:10:23.239752 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:10:23.239276    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f8cbff3-9dfd-4406-9c22-260bd366d946-metrics-tls podName:8f8cbff3-9dfd-4406-9c22-260bd366d946 nodeName:}" failed. No retries permitted until 2026-09-09 16:11:27.239252895 +0000 UTC m=+162.657102890 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8f8cbff3-9dfd-4406-9c22-260bd366d946-metrics-tls") pod "dns-default-frbps" (UID: "8f8cbff3-9dfd-4406-9c22-260bd366d946") : secret "dns-default-metrics-tls" not found
Sep 09 16:10:28.298880 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:10:28.298848    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-p4sh4"
Sep 09 16:10:53.849062 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:10:53.849009    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73b8ff58-aa37-4fab-9632-9b0f5d409902-metrics-certs\") pod \"network-metrics-daemon-t29nl\" (UID: \"73b8ff58-aa37-4fab-9632-9b0f5d409902\") " pod="openshift-multus/network-metrics-daemon-t29nl"
Sep 09 16:10:53.849582 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:10:53.849164    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 09 16:10:53.849582 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:10:53.849235    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73b8ff58-aa37-4fab-9632-9b0f5d409902-metrics-certs podName:73b8ff58-aa37-4fab-9632-9b0f5d409902 nodeName:}" failed. No retries permitted until 2026-09-09 16:12:55.849218698 +0000 UTC m=+251.267068674 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/73b8ff58-aa37-4fab-9632-9b0f5d409902-metrics-certs") pod "network-metrics-daemon-t29nl" (UID: "73b8ff58-aa37-4fab-9632-9b0f5d409902") : secret "metrics-daemon-secret" not found
Sep 09 16:11:22.416331 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:11:22.416280    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-frbps" podUID="8f8cbff3-9dfd-4406-9c22-260bd366d946"
Sep 09 16:11:22.497445 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:11:22.497420    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-frbps"
Sep 09 16:11:24.047923 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:11:24.047880    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-t29nl" podUID="73b8ff58-aa37-4fab-9632-9b0f5d409902"
Sep 09 16:11:27.287957 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:11:27.287895    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8f8cbff3-9dfd-4406-9c22-260bd366d946-metrics-tls\") pod \"dns-default-frbps\" (UID: \"8f8cbff3-9dfd-4406-9c22-260bd366d946\") " pod="openshift-dns/dns-default-frbps"
Sep 09 16:11:27.288368 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:11:27.288051    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 16:11:27.288368 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:11:27.288116    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f8cbff3-9dfd-4406-9c22-260bd366d946-metrics-tls podName:8f8cbff3-9dfd-4406-9c22-260bd366d946 nodeName:}" failed. No retries permitted until 2026-09-09 16:13:29.288098574 +0000 UTC m=+284.705948554 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8f8cbff3-9dfd-4406-9c22-260bd366d946-metrics-tls") pod "dns-default-frbps" (UID: "8f8cbff3-9dfd-4406-9c22-260bd366d946") : secret "dns-default-metrics-tls" not found
Sep 09 16:11:38.536629 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:11:38.536593    2567 generic.go:356] "Generic (PLEG): container finished" podID="bf814534-30a8-4914-9651-3f3ceb4e8e27" containerID="7215b96da9c2c10ebf6d6092fc1ee29e8b0b30542f94c889524071e6f972fbda" exitCode=255
Sep 09 16:11:38.537067 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:11:38.536666    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b8f54bb66-xffhq" event={"ID":"bf814534-30a8-4914-9651-3f3ceb4e8e27","Type":"ContainerDied","Data":"7215b96da9c2c10ebf6d6092fc1ee29e8b0b30542f94c889524071e6f972fbda"}
Sep 09 16:11:38.542434 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:11:38.542414    2567 scope.go:117] "RemoveContainer" containerID="7215b96da9c2c10ebf6d6092fc1ee29e8b0b30542f94c889524071e6f972fbda"
Sep 09 16:11:39.029806 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:11:39.029767    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t29nl"
Sep 09 16:11:39.541394 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:11:39.541359    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b8f54bb66-xffhq" event={"ID":"bf814534-30a8-4914-9651-3f3ceb4e8e27","Type":"ContainerStarted","Data":"1b691633fc7cdb446fc67862d4852ae2abbb746969aaac69cd727b2139e9ddaf"}
Sep 09 16:12:51.029245 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.029212    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-7d7756ccd6-mfpk8"]
Sep 09 16:12:51.032170 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.032150    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7d7756ccd6-mfpk8"
Sep 09 16:12:51.034261 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.034242    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 09 16:12:51.034418 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.034399    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 09 16:12:51.034522 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.034245    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 09 16:12:51.035782 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.035740    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 09 16:12:51.035928 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.035912    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-wm6fv\""
Sep 09 16:12:51.036005 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.035958    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 09 16:12:51.036204 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.036189    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 09 16:12:51.047793 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.047772    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7d7756ccd6-mfpk8"]
Sep 09 16:12:51.124568 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.124515    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-pxwsx"]
Sep 09 16:12:51.127418 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.127401    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7ccf8cd948-76htb"]
Sep 09 16:12:51.127556 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.127537    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-pxwsx"
Sep 09 16:12:51.129707 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.129682    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-sg4dg\""
Sep 09 16:12:51.129952 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.129931    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 09 16:12:51.130063 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.129955    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 09 16:12:51.130063 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.130021    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 09 16:12:51.130063 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.130045    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-85b454874b-xwlzs"]
Sep 09 16:12:51.130204 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.130189    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7ccf8cd948-76htb"
Sep 09 16:12:51.132612 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.132592    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-6jv6x\""
Sep 09 16:12:51.132715 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.132687    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 09 16:12:51.132777 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.132737    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-xwlzs"
Sep 09 16:12:51.132886 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.132868    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 09 16:12:51.133081 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.132954    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 09 16:12:51.135943 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.135925    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 09 16:12:51.136043 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.135926    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 09 16:12:51.136043 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.136030    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-wckvf\""
Sep 09 16:12:51.136681 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.136667    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 09 16:12:51.137096 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.137079    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 09 16:12:51.140561 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.140541    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-pxwsx"]
Sep 09 16:12:51.142284 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.142264    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 09 16:12:51.143656 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.143638    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 09 16:12:51.157589 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.157568    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7ccf8cd948-76htb"]
Sep 09 16:12:51.160731 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.160711    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ae67a1e-f25a-4218-a9ce-d4614dd65d43-service-ca-bundle\") pod \"router-default-7d7756ccd6-mfpk8\" (UID: \"5ae67a1e-f25a-4218-a9ce-d4614dd65d43\") " pod="openshift-ingress/router-default-7d7756ccd6-mfpk8"
Sep 09 16:12:51.160834 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.160743    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7c94p\" (UniqueName: \"kubernetes.io/projected/5ae67a1e-f25a-4218-a9ce-d4614dd65d43-kube-api-access-7c94p\") pod \"router-default-7d7756ccd6-mfpk8\" (UID: \"5ae67a1e-f25a-4218-a9ce-d4614dd65d43\") " pod="openshift-ingress/router-default-7d7756ccd6-mfpk8"
Sep 09 16:12:51.160834 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.160769    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/5ae67a1e-f25a-4218-a9ce-d4614dd65d43-stats-auth\") pod \"router-default-7d7756ccd6-mfpk8\" (UID: \"5ae67a1e-f25a-4218-a9ce-d4614dd65d43\") " pod="openshift-ingress/router-default-7d7756ccd6-mfpk8"
Sep 09 16:12:51.160943 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.160904    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/5ae67a1e-f25a-4218-a9ce-d4614dd65d43-default-certificate\") pod \"router-default-7d7756ccd6-mfpk8\" (UID: \"5ae67a1e-f25a-4218-a9ce-d4614dd65d43\") " pod="openshift-ingress/router-default-7d7756ccd6-mfpk8"
Sep 09 16:12:51.160943 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.160929    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ae67a1e-f25a-4218-a9ce-d4614dd65d43-metrics-certs\") pod \"router-default-7d7756ccd6-mfpk8\" (UID: \"5ae67a1e-f25a-4218-a9ce-d4614dd65d43\") " pod="openshift-ingress/router-default-7d7756ccd6-mfpk8"
Sep 09 16:12:51.176561 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.176531    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-xwlzs"]
Sep 09 16:12:51.261537 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.261498    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7c94p\" (UniqueName: \"kubernetes.io/projected/5ae67a1e-f25a-4218-a9ce-d4614dd65d43-kube-api-access-7c94p\") pod \"router-default-7d7756ccd6-mfpk8\" (UID: \"5ae67a1e-f25a-4218-a9ce-d4614dd65d43\") " pod="openshift-ingress/router-default-7d7756ccd6-mfpk8"
Sep 09 16:12:51.261537 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.261539    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/16ecfb4d-7944-465e-a38e-d8c5bda6b1ac-trusted-ca\") pod \"console-operator-85b454874b-xwlzs\" (UID: \"16ecfb4d-7944-465e-a38e-d8c5bda6b1ac\") " pod="openshift-console-operator/console-operator-85b454874b-xwlzs"
Sep 09 16:12:51.261761 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.261559    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v96fv\" (UniqueName: \"kubernetes.io/projected/16ecfb4d-7944-465e-a38e-d8c5bda6b1ac-kube-api-access-v96fv\") pod \"console-operator-85b454874b-xwlzs\" (UID: \"16ecfb4d-7944-465e-a38e-d8c5bda6b1ac\") " pod="openshift-console-operator/console-operator-85b454874b-xwlzs"
Sep 09 16:12:51.261761 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.261587    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/071d80b2-9f57-45c3-980d-678dc98590f1-registry-tls\") pod \"image-registry-7ccf8cd948-76htb\" (UID: \"071d80b2-9f57-45c3-980d-678dc98590f1\") " pod="openshift-image-registry/image-registry-7ccf8cd948-76htb"
Sep 09 16:12:51.261761 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.261618    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/5ae67a1e-f25a-4218-a9ce-d4614dd65d43-stats-auth\") pod \"router-default-7d7756ccd6-mfpk8\" (UID: \"5ae67a1e-f25a-4218-a9ce-d4614dd65d43\") " pod="openshift-ingress/router-default-7d7756ccd6-mfpk8"
Sep 09 16:12:51.261761 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.261647    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/071d80b2-9f57-45c3-980d-678dc98590f1-registry-certificates\") pod \"image-registry-7ccf8cd948-76htb\" (UID: \"071d80b2-9f57-45c3-980d-678dc98590f1\") " pod="openshift-image-registry/image-registry-7ccf8cd948-76htb"
Sep 09 16:12:51.261937 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.261794    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/16ecfb4d-7944-465e-a38e-d8c5bda6b1ac-config\") pod \"console-operator-85b454874b-xwlzs\" (UID: \"16ecfb4d-7944-465e-a38e-d8c5bda6b1ac\") " pod="openshift-console-operator/console-operator-85b454874b-xwlzs"
Sep 09 16:12:51.261937 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.261819    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/071d80b2-9f57-45c3-980d-678dc98590f1-installation-pull-secrets\") pod \"image-registry-7ccf8cd948-76htb\" (UID: \"071d80b2-9f57-45c3-980d-678dc98590f1\") " pod="openshift-image-registry/image-registry-7ccf8cd948-76htb"
Sep 09 16:12:51.261937 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.261855    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nszpg\" (UniqueName: \"kubernetes.io/projected/de20207a-fe5a-4afd-854b-3af462d9572f-kube-api-access-nszpg\") pod \"cluster-samples-operator-7cdcfcf76c-pxwsx\" (UID: \"de20207a-fe5a-4afd-854b-3af462d9572f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-pxwsx"
Sep 09 16:12:51.261937 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.261901    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/071d80b2-9f57-45c3-980d-678dc98590f1-image-registry-private-configuration\") pod \"image-registry-7ccf8cd948-76htb\" (UID: \"071d80b2-9f57-45c3-980d-678dc98590f1\") " pod="openshift-image-registry/image-registry-7ccf8cd948-76htb"
Sep 09 16:12:51.262133 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.261956    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/5ae67a1e-f25a-4218-a9ce-d4614dd65d43-default-certificate\") pod \"router-default-7d7756ccd6-mfpk8\" (UID: \"5ae67a1e-f25a-4218-a9ce-d4614dd65d43\") " pod="openshift-ingress/router-default-7d7756ccd6-mfpk8"
Sep 09 16:12:51.262133 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.261986    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ae67a1e-f25a-4218-a9ce-d4614dd65d43-metrics-certs\") pod \"router-default-7d7756ccd6-mfpk8\" (UID: \"5ae67a1e-f25a-4218-a9ce-d4614dd65d43\") " pod="openshift-ingress/router-default-7d7756ccd6-mfpk8"
Sep 09 16:12:51.262133 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.262010    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/16ecfb4d-7944-465e-a38e-d8c5bda6b1ac-serving-cert\") pod \"console-operator-85b454874b-xwlzs\" (UID: \"16ecfb4d-7944-465e-a38e-d8c5bda6b1ac\") " pod="openshift-console-operator/console-operator-85b454874b-xwlzs"
Sep 09 16:12:51.262133 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.262044    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/071d80b2-9f57-45c3-980d-678dc98590f1-ca-trust-extracted\") pod \"image-registry-7ccf8cd948-76htb\" (UID: \"071d80b2-9f57-45c3-980d-678dc98590f1\") " pod="openshift-image-registry/image-registry-7ccf8cd948-76htb"
Sep 09 16:12:51.262133 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.262069    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/071d80b2-9f57-45c3-980d-678dc98590f1-trusted-ca\") pod \"image-registry-7ccf8cd948-76htb\" (UID: \"071d80b2-9f57-45c3-980d-678dc98590f1\") " pod="openshift-image-registry/image-registry-7ccf8cd948-76htb"
Sep 09 16:12:51.262133 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:12:51.262088    2567 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 09 16:12:51.262133 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.262094    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdxvf\" (UniqueName: \"kubernetes.io/projected/071d80b2-9f57-45c3-980d-678dc98590f1-kube-api-access-fdxvf\") pod \"image-registry-7ccf8cd948-76htb\" (UID: \"071d80b2-9f57-45c3-980d-678dc98590f1\") " pod="openshift-image-registry/image-registry-7ccf8cd948-76htb"
Sep 09 16:12:51.262404 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:12:51.262186    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ae67a1e-f25a-4218-a9ce-d4614dd65d43-metrics-certs podName:5ae67a1e-f25a-4218-a9ce-d4614dd65d43 nodeName:}" failed. No retries permitted until 2026-09-09 16:12:51.762164899 +0000 UTC m=+247.180014878 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5ae67a1e-f25a-4218-a9ce-d4614dd65d43-metrics-certs") pod "router-default-7d7756ccd6-mfpk8" (UID: "5ae67a1e-f25a-4218-a9ce-d4614dd65d43") : secret "router-metrics-certs-default" not found
Sep 09 16:12:51.262404 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.262248    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ae67a1e-f25a-4218-a9ce-d4614dd65d43-service-ca-bundle\") pod \"router-default-7d7756ccd6-mfpk8\" (UID: \"5ae67a1e-f25a-4218-a9ce-d4614dd65d43\") " pod="openshift-ingress/router-default-7d7756ccd6-mfpk8"
Sep 09 16:12:51.262404 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.262281    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/de20207a-fe5a-4afd-854b-3af462d9572f-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-pxwsx\" (UID: \"de20207a-fe5a-4afd-854b-3af462d9572f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-pxwsx"
Sep 09 16:12:51.262404 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.262316    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/071d80b2-9f57-45c3-980d-678dc98590f1-bound-sa-token\") pod \"image-registry-7ccf8cd948-76htb\" (UID: \"071d80b2-9f57-45c3-980d-678dc98590f1\") " pod="openshift-image-registry/image-registry-7ccf8cd948-76htb"
Sep 09 16:12:51.262404 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:12:51.262394    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5ae67a1e-f25a-4218-a9ce-d4614dd65d43-service-ca-bundle podName:5ae67a1e-f25a-4218-a9ce-d4614dd65d43 nodeName:}" failed. No retries permitted until 2026-09-09 16:12:51.762374973 +0000 UTC m=+247.180224949 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5ae67a1e-f25a-4218-a9ce-d4614dd65d43-service-ca-bundle") pod "router-default-7d7756ccd6-mfpk8" (UID: "5ae67a1e-f25a-4218-a9ce-d4614dd65d43") : configmap references non-existent config key: service-ca.crt
Sep 09 16:12:51.264306 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.264281    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/5ae67a1e-f25a-4218-a9ce-d4614dd65d43-stats-auth\") pod \"router-default-7d7756ccd6-mfpk8\" (UID: \"5ae67a1e-f25a-4218-a9ce-d4614dd65d43\") " pod="openshift-ingress/router-default-7d7756ccd6-mfpk8"
Sep 09 16:12:51.264400 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.264343    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/5ae67a1e-f25a-4218-a9ce-d4614dd65d43-default-certificate\") pod \"router-default-7d7756ccd6-mfpk8\" (UID: \"5ae67a1e-f25a-4218-a9ce-d4614dd65d43\") " pod="openshift-ingress/router-default-7d7756ccd6-mfpk8"
Sep 09 16:12:51.269876 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.269857    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7c94p\" (UniqueName: \"kubernetes.io/projected/5ae67a1e-f25a-4218-a9ce-d4614dd65d43-kube-api-access-7c94p\") pod \"router-default-7d7756ccd6-mfpk8\" (UID: \"5ae67a1e-f25a-4218-a9ce-d4614dd65d43\") " pod="openshift-ingress/router-default-7d7756ccd6-mfpk8"
Sep 09 16:12:51.363201 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.363166    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/de20207a-fe5a-4afd-854b-3af462d9572f-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-pxwsx\" (UID: \"de20207a-fe5a-4afd-854b-3af462d9572f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-pxwsx"
Sep 09 16:12:51.363201 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.363206    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/071d80b2-9f57-45c3-980d-678dc98590f1-bound-sa-token\") pod \"image-registry-7ccf8cd948-76htb\" (UID: \"071d80b2-9f57-45c3-980d-678dc98590f1\") " pod="openshift-image-registry/image-registry-7ccf8cd948-76htb"
Sep 09 16:12:51.363506 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.363225    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/16ecfb4d-7944-465e-a38e-d8c5bda6b1ac-trusted-ca\") pod \"console-operator-85b454874b-xwlzs\" (UID: \"16ecfb4d-7944-465e-a38e-d8c5bda6b1ac\") " pod="openshift-console-operator/console-operator-85b454874b-xwlzs"
Sep 09 16:12:51.363506 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:12:51.363333    2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 09 16:12:51.363506 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.363362    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v96fv\" (UniqueName: \"kubernetes.io/projected/16ecfb4d-7944-465e-a38e-d8c5bda6b1ac-kube-api-access-v96fv\") pod \"console-operator-85b454874b-xwlzs\" (UID: \"16ecfb4d-7944-465e-a38e-d8c5bda6b1ac\") " pod="openshift-console-operator/console-operator-85b454874b-xwlzs"
Sep 09 16:12:51.363506 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:12:51.363421    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de20207a-fe5a-4afd-854b-3af462d9572f-samples-operator-tls podName:de20207a-fe5a-4afd-854b-3af462d9572f nodeName:}" failed. No retries permitted until 2026-09-09 16:12:51.86339883 +0000 UTC m=+247.281248822 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/de20207a-fe5a-4afd-854b-3af462d9572f-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-pxwsx" (UID: "de20207a-fe5a-4afd-854b-3af462d9572f") : secret "samples-operator-tls" not found
Sep 09 16:12:51.363506 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.363445    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/071d80b2-9f57-45c3-980d-678dc98590f1-registry-tls\") pod \"image-registry-7ccf8cd948-76htb\" (UID: \"071d80b2-9f57-45c3-980d-678dc98590f1\") " pod="openshift-image-registry/image-registry-7ccf8cd948-76htb"
Sep 09 16:12:51.363506 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.363506    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/071d80b2-9f57-45c3-980d-678dc98590f1-registry-certificates\") pod \"image-registry-7ccf8cd948-76htb\" (UID: \"071d80b2-9f57-45c3-980d-678dc98590f1\") " pod="openshift-image-registry/image-registry-7ccf8cd948-76htb"
Sep 09 16:12:51.363816 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.363558    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/16ecfb4d-7944-465e-a38e-d8c5bda6b1ac-config\") pod \"console-operator-85b454874b-xwlzs\" (UID: \"16ecfb4d-7944-465e-a38e-d8c5bda6b1ac\") " pod="openshift-console-operator/console-operator-85b454874b-xwlzs"
Sep 09 16:12:51.363816 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.363608    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/071d80b2-9f57-45c3-980d-678dc98590f1-installation-pull-secrets\") pod \"image-registry-7ccf8cd948-76htb\" (UID: \"071d80b2-9f57-45c3-980d-678dc98590f1\") " pod="openshift-image-registry/image-registry-7ccf8cd948-76htb"
Sep 09 16:12:51.363816 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:12:51.363612    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 16:12:51.363816 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:12:51.363631    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7ccf8cd948-76htb: secret "image-registry-tls" not found
Sep 09 16:12:51.363816 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.363667    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nszpg\" (UniqueName: \"kubernetes.io/projected/de20207a-fe5a-4afd-854b-3af462d9572f-kube-api-access-nszpg\") pod \"cluster-samples-operator-7cdcfcf76c-pxwsx\" (UID: \"de20207a-fe5a-4afd-854b-3af462d9572f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-pxwsx"
Sep 09 16:12:51.363816 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:12:51.363706    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/071d80b2-9f57-45c3-980d-678dc98590f1-registry-tls podName:071d80b2-9f57-45c3-980d-678dc98590f1 nodeName:}" failed. No retries permitted until 2026-09-09 16:12:51.863680275 +0000 UTC m=+247.281530268 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/071d80b2-9f57-45c3-980d-678dc98590f1-registry-tls") pod "image-registry-7ccf8cd948-76htb" (UID: "071d80b2-9f57-45c3-980d-678dc98590f1") : secret "image-registry-tls" not found
Sep 09 16:12:51.363816 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.363728    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/071d80b2-9f57-45c3-980d-678dc98590f1-image-registry-private-configuration\") pod \"image-registry-7ccf8cd948-76htb\" (UID: \"071d80b2-9f57-45c3-980d-678dc98590f1\") " pod="openshift-image-registry/image-registry-7ccf8cd948-76htb"
Sep 09 16:12:51.363816 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.363783    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/16ecfb4d-7944-465e-a38e-d8c5bda6b1ac-serving-cert\") pod \"console-operator-85b454874b-xwlzs\" (UID: \"16ecfb4d-7944-465e-a38e-d8c5bda6b1ac\") " pod="openshift-console-operator/console-operator-85b454874b-xwlzs"
Sep 09 16:12:51.363816 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.363814    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/071d80b2-9f57-45c3-980d-678dc98590f1-ca-trust-extracted\") pod \"image-registry-7ccf8cd948-76htb\" (UID: \"071d80b2-9f57-45c3-980d-678dc98590f1\") " pod="openshift-image-registry/image-registry-7ccf8cd948-76htb"
Sep 09 16:12:51.364245 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.363837    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/071d80b2-9f57-45c3-980d-678dc98590f1-trusted-ca\") pod \"image-registry-7ccf8cd948-76htb\" (UID: \"071d80b2-9f57-45c3-980d-678dc98590f1\") " pod="openshift-image-registry/image-registry-7ccf8cd948-76htb"
Sep 09 16:12:51.364245 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.363860    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdxvf\" (UniqueName: \"kubernetes.io/projected/071d80b2-9f57-45c3-980d-678dc98590f1-kube-api-access-fdxvf\") pod \"image-registry-7ccf8cd948-76htb\" (UID: \"071d80b2-9f57-45c3-980d-678dc98590f1\") " pod="openshift-image-registry/image-registry-7ccf8cd948-76htb"
Sep 09 16:12:51.364245 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.364192    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/071d80b2-9f57-45c3-980d-678dc98590f1-ca-trust-extracted\") pod \"image-registry-7ccf8cd948-76htb\" (UID: \"071d80b2-9f57-45c3-980d-678dc98590f1\") " pod="openshift-image-registry/image-registry-7ccf8cd948-76htb"
Sep 09 16:12:51.364245 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.364199    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/16ecfb4d-7944-465e-a38e-d8c5bda6b1ac-trusted-ca\") pod \"console-operator-85b454874b-xwlzs\" (UID: \"16ecfb4d-7944-465e-a38e-d8c5bda6b1ac\") " pod="openshift-console-operator/console-operator-85b454874b-xwlzs"
Sep 09 16:12:51.364245 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.364192    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/071d80b2-9f57-45c3-980d-678dc98590f1-registry-certificates\") pod \"image-registry-7ccf8cd948-76htb\" (UID: \"071d80b2-9f57-45c3-980d-678dc98590f1\") " pod="openshift-image-registry/image-registry-7ccf8cd948-76htb"
Sep 09 16:12:51.364245 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.364222    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/16ecfb4d-7944-465e-a38e-d8c5bda6b1ac-config\") pod \"console-operator-85b454874b-xwlzs\" (UID: \"16ecfb4d-7944-465e-a38e-d8c5bda6b1ac\") " pod="openshift-console-operator/console-operator-85b454874b-xwlzs"
Sep 09 16:12:51.365293 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.365263    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/071d80b2-9f57-45c3-980d-678dc98590f1-trusted-ca\") pod \"image-registry-7ccf8cd948-76htb\" (UID: \"071d80b2-9f57-45c3-980d-678dc98590f1\") " pod="openshift-image-registry/image-registry-7ccf8cd948-76htb"
Sep 09 16:12:51.366207 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.366178    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/071d80b2-9f57-45c3-980d-678dc98590f1-image-registry-private-configuration\") pod \"image-registry-7ccf8cd948-76htb\" (UID: \"071d80b2-9f57-45c3-980d-678dc98590f1\") " pod="openshift-image-registry/image-registry-7ccf8cd948-76htb"
Sep 09 16:12:51.366365 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.366348    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/16ecfb4d-7944-465e-a38e-d8c5bda6b1ac-serving-cert\") pod \"console-operator-85b454874b-xwlzs\" (UID: \"16ecfb4d-7944-465e-a38e-d8c5bda6b1ac\") " pod="openshift-console-operator/console-operator-85b454874b-xwlzs"
Sep 09 16:12:51.366410 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.366370    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/071d80b2-9f57-45c3-980d-678dc98590f1-installation-pull-secrets\") pod \"image-registry-7ccf8cd948-76htb\" (UID: \"071d80b2-9f57-45c3-980d-678dc98590f1\") " pod="openshift-image-registry/image-registry-7ccf8cd948-76htb"
Sep 09 16:12:51.372197 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.371928    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v96fv\" (UniqueName: \"kubernetes.io/projected/16ecfb4d-7944-465e-a38e-d8c5bda6b1ac-kube-api-access-v96fv\") pod \"console-operator-85b454874b-xwlzs\" (UID: \"16ecfb4d-7944-465e-a38e-d8c5bda6b1ac\") " pod="openshift-console-operator/console-operator-85b454874b-xwlzs"
Sep 09 16:12:51.372315 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.372281    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdxvf\" (UniqueName: \"kubernetes.io/projected/071d80b2-9f57-45c3-980d-678dc98590f1-kube-api-access-fdxvf\") pod \"image-registry-7ccf8cd948-76htb\" (UID: \"071d80b2-9f57-45c3-980d-678dc98590f1\") " pod="openshift-image-registry/image-registry-7ccf8cd948-76htb"
Sep 09 16:12:51.372386 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.372352    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nszpg\" (UniqueName: \"kubernetes.io/projected/de20207a-fe5a-4afd-854b-3af462d9572f-kube-api-access-nszpg\") pod \"cluster-samples-operator-7cdcfcf76c-pxwsx\" (UID: \"de20207a-fe5a-4afd-854b-3af462d9572f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-pxwsx"
Sep 09 16:12:51.372440 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.372407    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/071d80b2-9f57-45c3-980d-678dc98590f1-bound-sa-token\") pod \"image-registry-7ccf8cd948-76htb\" (UID: \"071d80b2-9f57-45c3-980d-678dc98590f1\") " pod="openshift-image-registry/image-registry-7ccf8cd948-76htb"
Sep 09 16:12:51.449967 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.449928    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-xwlzs"
Sep 09 16:12:51.719439 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.719361    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-xwlzs"]
Sep 09 16:12:51.722618 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:12:51.722585    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16ecfb4d_7944_465e_a38e_d8c5bda6b1ac.slice/crio-43e3c9e2afd796c9cb6952a3f41abd060488bf1353d9889c835fd3da1b3ba9f2 WatchSource:0}: Error finding container 43e3c9e2afd796c9cb6952a3f41abd060488bf1353d9889c835fd3da1b3ba9f2: Status 404 returned error can't find the container with id 43e3c9e2afd796c9cb6952a3f41abd060488bf1353d9889c835fd3da1b3ba9f2
Sep 09 16:12:51.765956 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.765915    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ae67a1e-f25a-4218-a9ce-d4614dd65d43-metrics-certs\") pod \"router-default-7d7756ccd6-mfpk8\" (UID: \"5ae67a1e-f25a-4218-a9ce-d4614dd65d43\") " pod="openshift-ingress/router-default-7d7756ccd6-mfpk8"
Sep 09 16:12:51.766064 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.765969    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ae67a1e-f25a-4218-a9ce-d4614dd65d43-service-ca-bundle\") pod \"router-default-7d7756ccd6-mfpk8\" (UID: \"5ae67a1e-f25a-4218-a9ce-d4614dd65d43\") " pod="openshift-ingress/router-default-7d7756ccd6-mfpk8"
Sep 09 16:12:51.766113 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:12:51.766058    2567 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 09 16:12:51.766113 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:12:51.766107    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ae67a1e-f25a-4218-a9ce-d4614dd65d43-metrics-certs podName:5ae67a1e-f25a-4218-a9ce-d4614dd65d43 nodeName:}" failed. No retries permitted until 2026-09-09 16:12:52.766092469 +0000 UTC m=+248.183942447 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5ae67a1e-f25a-4218-a9ce-d4614dd65d43-metrics-certs") pod "router-default-7d7756ccd6-mfpk8" (UID: "5ae67a1e-f25a-4218-a9ce-d4614dd65d43") : secret "router-metrics-certs-default" not found
Sep 09 16:12:51.766248 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:12:51.766184    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5ae67a1e-f25a-4218-a9ce-d4614dd65d43-service-ca-bundle podName:5ae67a1e-f25a-4218-a9ce-d4614dd65d43 nodeName:}" failed. No retries permitted until 2026-09-09 16:12:52.766177692 +0000 UTC m=+248.184027668 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5ae67a1e-f25a-4218-a9ce-d4614dd65d43-service-ca-bundle") pod "router-default-7d7756ccd6-mfpk8" (UID: "5ae67a1e-f25a-4218-a9ce-d4614dd65d43") : configmap references non-existent config key: service-ca.crt
Sep 09 16:12:51.866899 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.866861    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/de20207a-fe5a-4afd-854b-3af462d9572f-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-pxwsx\" (UID: \"de20207a-fe5a-4afd-854b-3af462d9572f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-pxwsx"
Sep 09 16:12:51.866899 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:51.866902    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/071d80b2-9f57-45c3-980d-678dc98590f1-registry-tls\") pod \"image-registry-7ccf8cd948-76htb\" (UID: \"071d80b2-9f57-45c3-980d-678dc98590f1\") " pod="openshift-image-registry/image-registry-7ccf8cd948-76htb"
Sep 09 16:12:51.867151 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:12:51.867035    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 16:12:51.867151 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:12:51.867031    2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 09 16:12:51.867151 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:12:51.867115    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de20207a-fe5a-4afd-854b-3af462d9572f-samples-operator-tls podName:de20207a-fe5a-4afd-854b-3af462d9572f nodeName:}" failed. No retries permitted until 2026-09-09 16:12:52.867096243 +0000 UTC m=+248.284946220 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/de20207a-fe5a-4afd-854b-3af462d9572f-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-pxwsx" (UID: "de20207a-fe5a-4afd-854b-3af462d9572f") : secret "samples-operator-tls" not found
Sep 09 16:12:51.867151 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:12:51.867046    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7ccf8cd948-76htb: secret "image-registry-tls" not found
Sep 09 16:12:51.867353 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:12:51.867179    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/071d80b2-9f57-45c3-980d-678dc98590f1-registry-tls podName:071d80b2-9f57-45c3-980d-678dc98590f1 nodeName:}" failed. No retries permitted until 2026-09-09 16:12:52.867165842 +0000 UTC m=+248.285015821 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/071d80b2-9f57-45c3-980d-678dc98590f1-registry-tls") pod "image-registry-7ccf8cd948-76htb" (UID: "071d80b2-9f57-45c3-980d-678dc98590f1") : secret "image-registry-tls" not found
Sep 09 16:12:52.725789 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:52.725752    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-xwlzs" event={"ID":"16ecfb4d-7944-465e-a38e-d8c5bda6b1ac","Type":"ContainerStarted","Data":"43e3c9e2afd796c9cb6952a3f41abd060488bf1353d9889c835fd3da1b3ba9f2"}
Sep 09 16:12:52.774496 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:52.774437    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ae67a1e-f25a-4218-a9ce-d4614dd65d43-metrics-certs\") pod \"router-default-7d7756ccd6-mfpk8\" (UID: \"5ae67a1e-f25a-4218-a9ce-d4614dd65d43\") " pod="openshift-ingress/router-default-7d7756ccd6-mfpk8"
Sep 09 16:12:52.774654 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:52.774512    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ae67a1e-f25a-4218-a9ce-d4614dd65d43-service-ca-bundle\") pod \"router-default-7d7756ccd6-mfpk8\" (UID: \"5ae67a1e-f25a-4218-a9ce-d4614dd65d43\") " pod="openshift-ingress/router-default-7d7756ccd6-mfpk8"
Sep 09 16:12:52.774654 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:12:52.774587    2567 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 09 16:12:52.774654 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:12:52.774650    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ae67a1e-f25a-4218-a9ce-d4614dd65d43-metrics-certs podName:5ae67a1e-f25a-4218-a9ce-d4614dd65d43 nodeName:}" failed. No retries permitted until 2026-09-09 16:12:54.774634565 +0000 UTC m=+250.192484542 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5ae67a1e-f25a-4218-a9ce-d4614dd65d43-metrics-certs") pod "router-default-7d7756ccd6-mfpk8" (UID: "5ae67a1e-f25a-4218-a9ce-d4614dd65d43") : secret "router-metrics-certs-default" not found
Sep 09 16:12:52.774778 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:12:52.774663    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5ae67a1e-f25a-4218-a9ce-d4614dd65d43-service-ca-bundle podName:5ae67a1e-f25a-4218-a9ce-d4614dd65d43 nodeName:}" failed. No retries permitted until 2026-09-09 16:12:54.774656824 +0000 UTC m=+250.192506801 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5ae67a1e-f25a-4218-a9ce-d4614dd65d43-service-ca-bundle") pod "router-default-7d7756ccd6-mfpk8" (UID: "5ae67a1e-f25a-4218-a9ce-d4614dd65d43") : configmap references non-existent config key: service-ca.crt
Sep 09 16:12:52.875397 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:52.875343    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/de20207a-fe5a-4afd-854b-3af462d9572f-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-pxwsx\" (UID: \"de20207a-fe5a-4afd-854b-3af462d9572f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-pxwsx"
Sep 09 16:12:52.875519 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:52.875406    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/071d80b2-9f57-45c3-980d-678dc98590f1-registry-tls\") pod \"image-registry-7ccf8cd948-76htb\" (UID: \"071d80b2-9f57-45c3-980d-678dc98590f1\") " pod="openshift-image-registry/image-registry-7ccf8cd948-76htb"
Sep 09 16:12:52.875582 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:12:52.875546    2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 09 16:12:52.875582 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:12:52.875557    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 16:12:52.875582 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:12:52.875572    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7ccf8cd948-76htb: secret "image-registry-tls" not found
Sep 09 16:12:52.875707 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:12:52.875627    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de20207a-fe5a-4afd-854b-3af462d9572f-samples-operator-tls podName:de20207a-fe5a-4afd-854b-3af462d9572f nodeName:}" failed. No retries permitted until 2026-09-09 16:12:54.875606461 +0000 UTC m=+250.293456469 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/de20207a-fe5a-4afd-854b-3af462d9572f-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-pxwsx" (UID: "de20207a-fe5a-4afd-854b-3af462d9572f") : secret "samples-operator-tls" not found
Sep 09 16:12:52.875707 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:12:52.875647    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/071d80b2-9f57-45c3-980d-678dc98590f1-registry-tls podName:071d80b2-9f57-45c3-980d-678dc98590f1 nodeName:}" failed. No retries permitted until 2026-09-09 16:12:54.875637625 +0000 UTC m=+250.293487622 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/071d80b2-9f57-45c3-980d-678dc98590f1-registry-tls") pod "image-registry-7ccf8cd948-76htb" (UID: "071d80b2-9f57-45c3-980d-678dc98590f1") : secret "image-registry-tls" not found
Sep 09 16:12:54.790671 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:54.790627    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ae67a1e-f25a-4218-a9ce-d4614dd65d43-metrics-certs\") pod \"router-default-7d7756ccd6-mfpk8\" (UID: \"5ae67a1e-f25a-4218-a9ce-d4614dd65d43\") " pod="openshift-ingress/router-default-7d7756ccd6-mfpk8"
Sep 09 16:12:54.791098 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:54.790694    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ae67a1e-f25a-4218-a9ce-d4614dd65d43-service-ca-bundle\") pod \"router-default-7d7756ccd6-mfpk8\" (UID: \"5ae67a1e-f25a-4218-a9ce-d4614dd65d43\") " pod="openshift-ingress/router-default-7d7756ccd6-mfpk8"
Sep 09 16:12:54.791098 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:12:54.790806    2567 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 09 16:12:54.791098 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:12:54.790823    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5ae67a1e-f25a-4218-a9ce-d4614dd65d43-service-ca-bundle podName:5ae67a1e-f25a-4218-a9ce-d4614dd65d43 nodeName:}" failed. No retries permitted until 2026-09-09 16:12:58.790803471 +0000 UTC m=+254.208653468 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5ae67a1e-f25a-4218-a9ce-d4614dd65d43-service-ca-bundle") pod "router-default-7d7756ccd6-mfpk8" (UID: "5ae67a1e-f25a-4218-a9ce-d4614dd65d43") : configmap references non-existent config key: service-ca.crt
Sep 09 16:12:54.791098 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:12:54.790868    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ae67a1e-f25a-4218-a9ce-d4614dd65d43-metrics-certs podName:5ae67a1e-f25a-4218-a9ce-d4614dd65d43 nodeName:}" failed. No retries permitted until 2026-09-09 16:12:58.790850883 +0000 UTC m=+254.208700865 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5ae67a1e-f25a-4218-a9ce-d4614dd65d43-metrics-certs") pod "router-default-7d7756ccd6-mfpk8" (UID: "5ae67a1e-f25a-4218-a9ce-d4614dd65d43") : secret "router-metrics-certs-default" not found
Sep 09 16:12:54.891760 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:54.891726    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/de20207a-fe5a-4afd-854b-3af462d9572f-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-pxwsx\" (UID: \"de20207a-fe5a-4afd-854b-3af462d9572f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-pxwsx"
Sep 09 16:12:54.891760 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:54.891763    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/071d80b2-9f57-45c3-980d-678dc98590f1-registry-tls\") pod \"image-registry-7ccf8cd948-76htb\" (UID: \"071d80b2-9f57-45c3-980d-678dc98590f1\") " pod="openshift-image-registry/image-registry-7ccf8cd948-76htb"
Sep 09 16:12:54.891996 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:12:54.891855    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 16:12:54.891996 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:12:54.891866    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7ccf8cd948-76htb: secret "image-registry-tls" not found
Sep 09 16:12:54.891996 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:12:54.891880    2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 09 16:12:54.891996 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:12:54.891912    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/071d80b2-9f57-45c3-980d-678dc98590f1-registry-tls podName:071d80b2-9f57-45c3-980d-678dc98590f1 nodeName:}" failed. No retries permitted until 2026-09-09 16:12:58.891899796 +0000 UTC m=+254.309749773 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/071d80b2-9f57-45c3-980d-678dc98590f1-registry-tls") pod "image-registry-7ccf8cd948-76htb" (UID: "071d80b2-9f57-45c3-980d-678dc98590f1") : secret "image-registry-tls" not found
Sep 09 16:12:54.891996 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:12:54.891939    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de20207a-fe5a-4afd-854b-3af462d9572f-samples-operator-tls podName:de20207a-fe5a-4afd-854b-3af462d9572f nodeName:}" failed. No retries permitted until 2026-09-09 16:12:58.891923251 +0000 UTC m=+254.309773228 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/de20207a-fe5a-4afd-854b-3af462d9572f-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-pxwsx" (UID: "de20207a-fe5a-4afd-854b-3af462d9572f") : secret "samples-operator-tls" not found
Sep 09 16:12:55.897958 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:55.897925    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73b8ff58-aa37-4fab-9632-9b0f5d409902-metrics-certs\") pod \"network-metrics-daemon-t29nl\" (UID: \"73b8ff58-aa37-4fab-9632-9b0f5d409902\") " pod="openshift-multus/network-metrics-daemon-t29nl"
Sep 09 16:12:55.898348 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:12:55.898082    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 09 16:12:55.898348 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:12:55.898163    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73b8ff58-aa37-4fab-9632-9b0f5d409902-metrics-certs podName:73b8ff58-aa37-4fab-9632-9b0f5d409902 nodeName:}" failed. No retries permitted until 2026-09-09 16:14:57.898146234 +0000 UTC m=+373.315996211 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/73b8ff58-aa37-4fab-9632-9b0f5d409902-metrics-certs") pod "network-metrics-daemon-t29nl" (UID: "73b8ff58-aa37-4fab-9632-9b0f5d409902") : secret "metrics-daemon-secret" not found
Sep 09 16:12:56.742775 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:56.742744    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-xwlzs_16ecfb4d-7944-465e-a38e-d8c5bda6b1ac/console-operator/0.log"
Sep 09 16:12:56.743013 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:56.742785    2567 generic.go:356] "Generic (PLEG): container finished" podID="16ecfb4d-7944-465e-a38e-d8c5bda6b1ac" containerID="10d9559d6a86c40e40b83d84496d698b9489a0afa3f42cb8a94f1af08aaa1940" exitCode=255
Sep 09 16:12:56.743013 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:56.742820    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-xwlzs" event={"ID":"16ecfb4d-7944-465e-a38e-d8c5bda6b1ac","Type":"ContainerDied","Data":"10d9559d6a86c40e40b83d84496d698b9489a0afa3f42cb8a94f1af08aaa1940"}
Sep 09 16:12:56.743084 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:56.743051    2567 scope.go:117] "RemoveContainer" containerID="10d9559d6a86c40e40b83d84496d698b9489a0afa3f42cb8a94f1af08aaa1940"
Sep 09 16:12:57.190339 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:57.190292    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-lg6v8"]
Sep 09 16:12:57.193537 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:57.193514    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lg6v8"
Sep 09 16:12:57.195565 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:57.195542    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 09 16:12:57.195686 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:57.195647    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 09 16:12:57.195918 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:57.195901    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 09 16:12:57.196324 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:57.196309    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-rd78t\""
Sep 09 16:12:57.196393 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:57.196312    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 09 16:12:57.204374 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:57.204350    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lg6v8"]
Sep 09 16:12:57.309099 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:57.309057    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2svk5\" (UniqueName: \"kubernetes.io/projected/cade2714-b3c7-4de0-841b-b2f3bc582e6c-kube-api-access-2svk5\") pod \"insights-runtime-extractor-lg6v8\" (UID: \"cade2714-b3c7-4de0-841b-b2f3bc582e6c\") " pod="openshift-insights/insights-runtime-extractor-lg6v8"
Sep 09 16:12:57.309271 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:57.309110    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cade2714-b3c7-4de0-841b-b2f3bc582e6c-crio-socket\") pod \"insights-runtime-extractor-lg6v8\" (UID: \"cade2714-b3c7-4de0-841b-b2f3bc582e6c\") " pod="openshift-insights/insights-runtime-extractor-lg6v8"
Sep 09 16:12:57.309271 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:57.309161    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cade2714-b3c7-4de0-841b-b2f3bc582e6c-data-volume\") pod \"insights-runtime-extractor-lg6v8\" (UID: \"cade2714-b3c7-4de0-841b-b2f3bc582e6c\") " pod="openshift-insights/insights-runtime-extractor-lg6v8"
Sep 09 16:12:57.309271 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:57.309200    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cade2714-b3c7-4de0-841b-b2f3bc582e6c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lg6v8\" (UID: \"cade2714-b3c7-4de0-841b-b2f3bc582e6c\") " pod="openshift-insights/insights-runtime-extractor-lg6v8"
Sep 09 16:12:57.309271 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:57.309235    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cade2714-b3c7-4de0-841b-b2f3bc582e6c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lg6v8\" (UID: \"cade2714-b3c7-4de0-841b-b2f3bc582e6c\") " pod="openshift-insights/insights-runtime-extractor-lg6v8"
Sep 09 16:12:57.410597 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:57.410545    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cade2714-b3c7-4de0-841b-b2f3bc582e6c-data-volume\") pod \"insights-runtime-extractor-lg6v8\" (UID: \"cade2714-b3c7-4de0-841b-b2f3bc582e6c\") " pod="openshift-insights/insights-runtime-extractor-lg6v8"
Sep 09 16:12:57.410769 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:57.410620    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cade2714-b3c7-4de0-841b-b2f3bc582e6c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lg6v8\" (UID: \"cade2714-b3c7-4de0-841b-b2f3bc582e6c\") " pod="openshift-insights/insights-runtime-extractor-lg6v8"
Sep 09 16:12:57.410769 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:57.410671    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cade2714-b3c7-4de0-841b-b2f3bc582e6c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lg6v8\" (UID: \"cade2714-b3c7-4de0-841b-b2f3bc582e6c\") " pod="openshift-insights/insights-runtime-extractor-lg6v8"
Sep 09 16:12:57.410769 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:57.410744    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2svk5\" (UniqueName: \"kubernetes.io/projected/cade2714-b3c7-4de0-841b-b2f3bc582e6c-kube-api-access-2svk5\") pod \"insights-runtime-extractor-lg6v8\" (UID: \"cade2714-b3c7-4de0-841b-b2f3bc582e6c\") " pod="openshift-insights/insights-runtime-extractor-lg6v8"
Sep 09 16:12:57.410914 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:57.410776    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cade2714-b3c7-4de0-841b-b2f3bc582e6c-crio-socket\") pod \"insights-runtime-extractor-lg6v8\" (UID: \"cade2714-b3c7-4de0-841b-b2f3bc582e6c\") " pod="openshift-insights/insights-runtime-extractor-lg6v8"
Sep 09 16:12:57.410914 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:12:57.410776    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 16:12:57.410914 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:57.410845    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cade2714-b3c7-4de0-841b-b2f3bc582e6c-crio-socket\") pod \"insights-runtime-extractor-lg6v8\" (UID: \"cade2714-b3c7-4de0-841b-b2f3bc582e6c\") " pod="openshift-insights/insights-runtime-extractor-lg6v8"
Sep 09 16:12:57.410914 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:57.410877    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cade2714-b3c7-4de0-841b-b2f3bc582e6c-data-volume\") pod \"insights-runtime-extractor-lg6v8\" (UID: \"cade2714-b3c7-4de0-841b-b2f3bc582e6c\") " pod="openshift-insights/insights-runtime-extractor-lg6v8"
Sep 09 16:12:57.410914 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:12:57.410829    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 16:12:57.411101 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:12:57.410898    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cade2714-b3c7-4de0-841b-b2f3bc582e6c-insights-runtime-extractor-tls podName:cade2714-b3c7-4de0-841b-b2f3bc582e6c nodeName:}" failed. No retries permitted until 2026-09-09 16:12:57.910878139 +0000 UTC m=+253.328728139 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cade2714-b3c7-4de0-841b-b2f3bc582e6c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lg6v8" (UID: "cade2714-b3c7-4de0-841b-b2f3bc582e6c") : secret "insights-runtime-extractor-tls" not found
Sep 09 16:12:57.411101 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:12:57.410961    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cade2714-b3c7-4de0-841b-b2f3bc582e6c-kube-rbac-proxy-cm podName:cade2714-b3c7-4de0-841b-b2f3bc582e6c nodeName:}" failed. No retries permitted until 2026-09-09 16:12:57.910947815 +0000 UTC m=+253.328797796 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/cade2714-b3c7-4de0-841b-b2f3bc582e6c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lg6v8" (UID: "cade2714-b3c7-4de0-841b-b2f3bc582e6c") : configmap "kube-rbac-proxy" not found
Sep 09 16:12:57.418597 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:57.418573    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2svk5\" (UniqueName: \"kubernetes.io/projected/cade2714-b3c7-4de0-841b-b2f3bc582e6c-kube-api-access-2svk5\") pod \"insights-runtime-extractor-lg6v8\" (UID: \"cade2714-b3c7-4de0-841b-b2f3bc582e6c\") " pod="openshift-insights/insights-runtime-extractor-lg6v8"
Sep 09 16:12:57.748009 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:57.747981    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-xwlzs_16ecfb4d-7944-465e-a38e-d8c5bda6b1ac/console-operator/1.log"
Sep 09 16:12:57.748419 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:57.748398    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-xwlzs_16ecfb4d-7944-465e-a38e-d8c5bda6b1ac/console-operator/0.log"
Sep 09 16:12:57.748558 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:57.748441    2567 generic.go:356] "Generic (PLEG): container finished" podID="16ecfb4d-7944-465e-a38e-d8c5bda6b1ac" containerID="b8b9072853592f000cfe83bd43871e328b5efa114b2a093d70a6a1a6d7a32f33" exitCode=255
Sep 09 16:12:57.748558 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:57.748506    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-xwlzs" event={"ID":"16ecfb4d-7944-465e-a38e-d8c5bda6b1ac","Type":"ContainerDied","Data":"b8b9072853592f000cfe83bd43871e328b5efa114b2a093d70a6a1a6d7a32f33"}
Sep 09 16:12:57.748665 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:57.748562    2567 scope.go:117] "RemoveContainer" containerID="10d9559d6a86c40e40b83d84496d698b9489a0afa3f42cb8a94f1af08aaa1940"
Sep 09 16:12:57.748722 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:57.748709    2567 scope.go:117] "RemoveContainer" containerID="b8b9072853592f000cfe83bd43871e328b5efa114b2a093d70a6a1a6d7a32f33"
Sep 09 16:12:57.748915 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:12:57.748892    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-xwlzs_openshift-console-operator(16ecfb4d-7944-465e-a38e-d8c5bda6b1ac)\"" pod="openshift-console-operator/console-operator-85b454874b-xwlzs" podUID="16ecfb4d-7944-465e-a38e-d8c5bda6b1ac"
Sep 09 16:12:57.915280 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:57.915225    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cade2714-b3c7-4de0-841b-b2f3bc582e6c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lg6v8\" (UID: \"cade2714-b3c7-4de0-841b-b2f3bc582e6c\") " pod="openshift-insights/insights-runtime-extractor-lg6v8"
Sep 09 16:12:57.915502 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:57.915299    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cade2714-b3c7-4de0-841b-b2f3bc582e6c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lg6v8\" (UID: \"cade2714-b3c7-4de0-841b-b2f3bc582e6c\") " pod="openshift-insights/insights-runtime-extractor-lg6v8"
Sep 09 16:12:57.915502 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:12:57.915367    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 16:12:57.915502 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:12:57.915397    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 16:12:57.915502 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:12:57.915429    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cade2714-b3c7-4de0-841b-b2f3bc582e6c-insights-runtime-extractor-tls podName:cade2714-b3c7-4de0-841b-b2f3bc582e6c nodeName:}" failed. No retries permitted until 2026-09-09 16:12:58.915414567 +0000 UTC m=+254.333264543 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cade2714-b3c7-4de0-841b-b2f3bc582e6c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lg6v8" (UID: "cade2714-b3c7-4de0-841b-b2f3bc582e6c") : secret "insights-runtime-extractor-tls" not found
Sep 09 16:12:57.915502 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:12:57.915446    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cade2714-b3c7-4de0-841b-b2f3bc582e6c-kube-rbac-proxy-cm podName:cade2714-b3c7-4de0-841b-b2f3bc582e6c nodeName:}" failed. No retries permitted until 2026-09-09 16:12:58.915438173 +0000 UTC m=+254.333288149 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/cade2714-b3c7-4de0-841b-b2f3bc582e6c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lg6v8" (UID: "cade2714-b3c7-4de0-841b-b2f3bc582e6c") : configmap "kube-rbac-proxy" not found
Sep 09 16:12:58.753424 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:58.753398    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-xwlzs_16ecfb4d-7944-465e-a38e-d8c5bda6b1ac/console-operator/1.log"
Sep 09 16:12:58.753803 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:58.753754    2567 scope.go:117] "RemoveContainer" containerID="b8b9072853592f000cfe83bd43871e328b5efa114b2a093d70a6a1a6d7a32f33"
Sep 09 16:12:58.753921 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:12:58.753903    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-xwlzs_openshift-console-operator(16ecfb4d-7944-465e-a38e-d8c5bda6b1ac)\"" pod="openshift-console-operator/console-operator-85b454874b-xwlzs" podUID="16ecfb4d-7944-465e-a38e-d8c5bda6b1ac"
Sep 09 16:12:58.821372 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:58.821325    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ae67a1e-f25a-4218-a9ce-d4614dd65d43-service-ca-bundle\") pod \"router-default-7d7756ccd6-mfpk8\" (UID: \"5ae67a1e-f25a-4218-a9ce-d4614dd65d43\") " pod="openshift-ingress/router-default-7d7756ccd6-mfpk8"
Sep 09 16:12:58.821608 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:58.821587    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ae67a1e-f25a-4218-a9ce-d4614dd65d43-metrics-certs\") pod \"router-default-7d7756ccd6-mfpk8\" (UID: \"5ae67a1e-f25a-4218-a9ce-d4614dd65d43\") " pod="openshift-ingress/router-default-7d7756ccd6-mfpk8"
Sep 09 16:12:58.821660 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:12:58.821610    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5ae67a1e-f25a-4218-a9ce-d4614dd65d43-service-ca-bundle podName:5ae67a1e-f25a-4218-a9ce-d4614dd65d43 nodeName:}" failed. No retries permitted until 2026-09-09 16:13:06.821580318 +0000 UTC m=+262.239430306 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5ae67a1e-f25a-4218-a9ce-d4614dd65d43-service-ca-bundle") pod "router-default-7d7756ccd6-mfpk8" (UID: "5ae67a1e-f25a-4218-a9ce-d4614dd65d43") : configmap references non-existent config key: service-ca.crt
Sep 09 16:12:58.821705 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:12:58.821673    2567 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 09 16:12:58.821751 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:12:58.821724    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ae67a1e-f25a-4218-a9ce-d4614dd65d43-metrics-certs podName:5ae67a1e-f25a-4218-a9ce-d4614dd65d43 nodeName:}" failed. No retries permitted until 2026-09-09 16:13:06.821707998 +0000 UTC m=+262.239557990 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5ae67a1e-f25a-4218-a9ce-d4614dd65d43-metrics-certs") pod "router-default-7d7756ccd6-mfpk8" (UID: "5ae67a1e-f25a-4218-a9ce-d4614dd65d43") : secret "router-metrics-certs-default" not found
Sep 09 16:12:58.922853 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:58.922818    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cade2714-b3c7-4de0-841b-b2f3bc582e6c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lg6v8\" (UID: \"cade2714-b3c7-4de0-841b-b2f3bc582e6c\") " pod="openshift-insights/insights-runtime-extractor-lg6v8"
Sep 09 16:12:58.922948 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:58.922857    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/de20207a-fe5a-4afd-854b-3af462d9572f-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-pxwsx\" (UID: \"de20207a-fe5a-4afd-854b-3af462d9572f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-pxwsx"
Sep 09 16:12:58.922948 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:58.922876    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/071d80b2-9f57-45c3-980d-678dc98590f1-registry-tls\") pod \"image-registry-7ccf8cd948-76htb\" (UID: \"071d80b2-9f57-45c3-980d-678dc98590f1\") " pod="openshift-image-registry/image-registry-7ccf8cd948-76htb"
Sep 09 16:12:58.922948 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:58.922895    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cade2714-b3c7-4de0-841b-b2f3bc582e6c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lg6v8\" (UID: \"cade2714-b3c7-4de0-841b-b2f3bc582e6c\") " pod="openshift-insights/insights-runtime-extractor-lg6v8"
Sep 09 16:12:58.923064 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:12:58.922962    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 16:12:58.923064 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:12:58.922974    2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 09 16:12:58.923064 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:12:58.922997    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 16:12:58.923064 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:12:58.923014    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7ccf8cd948-76htb: secret "image-registry-tls" not found
Sep 09 16:12:58.923064 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:12:58.923028    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cade2714-b3c7-4de0-841b-b2f3bc582e6c-insights-runtime-extractor-tls podName:cade2714-b3c7-4de0-841b-b2f3bc582e6c nodeName:}" failed. No retries permitted until 2026-09-09 16:13:00.923009801 +0000 UTC m=+256.340859795 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cade2714-b3c7-4de0-841b-b2f3bc582e6c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lg6v8" (UID: "cade2714-b3c7-4de0-841b-b2f3bc582e6c") : secret "insights-runtime-extractor-tls" not found
Sep 09 16:12:58.923064 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:12:58.923037    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 16:12:58.923064 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:12:58.923047    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/071d80b2-9f57-45c3-980d-678dc98590f1-registry-tls podName:071d80b2-9f57-45c3-980d-678dc98590f1 nodeName:}" failed. No retries permitted until 2026-09-09 16:13:06.923036548 +0000 UTC m=+262.340886526 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/071d80b2-9f57-45c3-980d-678dc98590f1-registry-tls") pod "image-registry-7ccf8cd948-76htb" (UID: "071d80b2-9f57-45c3-980d-678dc98590f1") : secret "image-registry-tls" not found
Sep 09 16:12:58.923064 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:12:58.923059    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de20207a-fe5a-4afd-854b-3af462d9572f-samples-operator-tls podName:de20207a-fe5a-4afd-854b-3af462d9572f nodeName:}" failed. No retries permitted until 2026-09-09 16:13:06.923053538 +0000 UTC m=+262.340903514 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/de20207a-fe5a-4afd-854b-3af462d9572f-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-pxwsx" (UID: "de20207a-fe5a-4afd-854b-3af462d9572f") : secret "samples-operator-tls" not found
Sep 09 16:12:58.923064 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:12:58.923070    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cade2714-b3c7-4de0-841b-b2f3bc582e6c-kube-rbac-proxy-cm podName:cade2714-b3c7-4de0-841b-b2f3bc582e6c nodeName:}" failed. No retries permitted until 2026-09-09 16:13:00.923064838 +0000 UTC m=+256.340914815 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/cade2714-b3c7-4de0-841b-b2f3bc582e6c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lg6v8" (UID: "cade2714-b3c7-4de0-841b-b2f3bc582e6c") : configmap "kube-rbac-proxy" not found
Sep 09 16:12:59.944279 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:12:59.944250    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-cnvzz_deb44589-97aa-464f-95b0-61362f120948/dns-node-resolver/0.log"
Sep 09 16:13:00.744276 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:00.744242    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-c7kvc_f996fce2-faf6-474a-997d-8a9419e788d8/node-ca/0.log"
Sep 09 16:13:00.938848 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:00.938794    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cade2714-b3c7-4de0-841b-b2f3bc582e6c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lg6v8\" (UID: \"cade2714-b3c7-4de0-841b-b2f3bc582e6c\") " pod="openshift-insights/insights-runtime-extractor-lg6v8"
Sep 09 16:13:00.939030 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:00.938880    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cade2714-b3c7-4de0-841b-b2f3bc582e6c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lg6v8\" (UID: \"cade2714-b3c7-4de0-841b-b2f3bc582e6c\") " pod="openshift-insights/insights-runtime-extractor-lg6v8"
Sep 09 16:13:00.939030 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:13:00.938948    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 16:13:00.939030 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:13:00.939007    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 16:13:00.939030 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:13:00.939017    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cade2714-b3c7-4de0-841b-b2f3bc582e6c-insights-runtime-extractor-tls podName:cade2714-b3c7-4de0-841b-b2f3bc582e6c nodeName:}" failed. No retries permitted until 2026-09-09 16:13:04.939001302 +0000 UTC m=+260.356851279 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cade2714-b3c7-4de0-841b-b2f3bc582e6c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lg6v8" (UID: "cade2714-b3c7-4de0-841b-b2f3bc582e6c") : secret "insights-runtime-extractor-tls" not found
Sep 09 16:13:00.939171 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:13:00.939056    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cade2714-b3c7-4de0-841b-b2f3bc582e6c-kube-rbac-proxy-cm podName:cade2714-b3c7-4de0-841b-b2f3bc582e6c nodeName:}" failed. No retries permitted until 2026-09-09 16:13:04.939043038 +0000 UTC m=+260.356893015 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/cade2714-b3c7-4de0-841b-b2f3bc582e6c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lg6v8" (UID: "cade2714-b3c7-4de0-841b-b2f3bc582e6c") : configmap "kube-rbac-proxy" not found
Sep 09 16:13:01.450580 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:01.450541    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-xwlzs"
Sep 09 16:13:01.450968 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:01.450611    2567 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-xwlzs"
Sep 09 16:13:01.450968 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:01.450914    2567 scope.go:117] "RemoveContainer" containerID="b8b9072853592f000cfe83bd43871e328b5efa114b2a093d70a6a1a6d7a32f33"
Sep 09 16:13:01.451084 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:13:01.451064    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-xwlzs_openshift-console-operator(16ecfb4d-7944-465e-a38e-d8c5bda6b1ac)\"" pod="openshift-console-operator/console-operator-85b454874b-xwlzs" podUID="16ecfb4d-7944-465e-a38e-d8c5bda6b1ac"
Sep 09 16:13:01.944416 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:01.944382    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tbdg5_65ebbfa4-b996-49d0-ae3e-2a1b8c3d7677/serve-healthcheck-canary/0.log"
Sep 09 16:13:04.971790 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:04.971758    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cade2714-b3c7-4de0-841b-b2f3bc582e6c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lg6v8\" (UID: \"cade2714-b3c7-4de0-841b-b2f3bc582e6c\") " pod="openshift-insights/insights-runtime-extractor-lg6v8"
Sep 09 16:13:04.972205 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:04.971821    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cade2714-b3c7-4de0-841b-b2f3bc582e6c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lg6v8\" (UID: \"cade2714-b3c7-4de0-841b-b2f3bc582e6c\") " pod="openshift-insights/insights-runtime-extractor-lg6v8"
Sep 09 16:13:04.972205 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:13:04.971927    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 16:13:04.972205 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:13:04.971947    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 16:13:04.972205 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:13:04.971995    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cade2714-b3c7-4de0-841b-b2f3bc582e6c-insights-runtime-extractor-tls podName:cade2714-b3c7-4de0-841b-b2f3bc582e6c nodeName:}" failed. No retries permitted until 2026-09-09 16:13:12.971979345 +0000 UTC m=+268.389829321 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cade2714-b3c7-4de0-841b-b2f3bc582e6c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lg6v8" (UID: "cade2714-b3c7-4de0-841b-b2f3bc582e6c") : secret "insights-runtime-extractor-tls" not found
Sep 09 16:13:04.972205 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:13:04.972009    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cade2714-b3c7-4de0-841b-b2f3bc582e6c-kube-rbac-proxy-cm podName:cade2714-b3c7-4de0-841b-b2f3bc582e6c nodeName:}" failed. No retries permitted until 2026-09-09 16:13:12.972002872 +0000 UTC m=+268.389852848 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/cade2714-b3c7-4de0-841b-b2f3bc582e6c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lg6v8" (UID: "cade2714-b3c7-4de0-841b-b2f3bc582e6c") : configmap "kube-rbac-proxy" not found
Sep 09 16:13:06.888169 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:06.888086    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ae67a1e-f25a-4218-a9ce-d4614dd65d43-service-ca-bundle\") pod \"router-default-7d7756ccd6-mfpk8\" (UID: \"5ae67a1e-f25a-4218-a9ce-d4614dd65d43\") " pod="openshift-ingress/router-default-7d7756ccd6-mfpk8"
Sep 09 16:13:06.888592 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:06.888191    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ae67a1e-f25a-4218-a9ce-d4614dd65d43-metrics-certs\") pod \"router-default-7d7756ccd6-mfpk8\" (UID: \"5ae67a1e-f25a-4218-a9ce-d4614dd65d43\") " pod="openshift-ingress/router-default-7d7756ccd6-mfpk8"
Sep 09 16:13:06.888592 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:13:06.888256    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5ae67a1e-f25a-4218-a9ce-d4614dd65d43-service-ca-bundle podName:5ae67a1e-f25a-4218-a9ce-d4614dd65d43 nodeName:}" failed. No retries permitted until 2026-09-09 16:13:22.888237375 +0000 UTC m=+278.306087352 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5ae67a1e-f25a-4218-a9ce-d4614dd65d43-service-ca-bundle") pod "router-default-7d7756ccd6-mfpk8" (UID: "5ae67a1e-f25a-4218-a9ce-d4614dd65d43") : configmap references non-existent config key: service-ca.crt
Sep 09 16:13:06.888592 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:13:06.888318    2567 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 09 16:13:06.888592 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:13:06.888391    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ae67a1e-f25a-4218-a9ce-d4614dd65d43-metrics-certs podName:5ae67a1e-f25a-4218-a9ce-d4614dd65d43 nodeName:}" failed. No retries permitted until 2026-09-09 16:13:22.888366971 +0000 UTC m=+278.306216961 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5ae67a1e-f25a-4218-a9ce-d4614dd65d43-metrics-certs") pod "router-default-7d7756ccd6-mfpk8" (UID: "5ae67a1e-f25a-4218-a9ce-d4614dd65d43") : secret "router-metrics-certs-default" not found
Sep 09 16:13:06.988745 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:06.988709    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/de20207a-fe5a-4afd-854b-3af462d9572f-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-pxwsx\" (UID: \"de20207a-fe5a-4afd-854b-3af462d9572f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-pxwsx"
Sep 09 16:13:06.988745 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:06.988749    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/071d80b2-9f57-45c3-980d-678dc98590f1-registry-tls\") pod \"image-registry-7ccf8cd948-76htb\" (UID: \"071d80b2-9f57-45c3-980d-678dc98590f1\") " pod="openshift-image-registry/image-registry-7ccf8cd948-76htb"
Sep 09 16:13:06.991329 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:06.991299    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/071d80b2-9f57-45c3-980d-678dc98590f1-registry-tls\") pod \"image-registry-7ccf8cd948-76htb\" (UID: \"071d80b2-9f57-45c3-980d-678dc98590f1\") " pod="openshift-image-registry/image-registry-7ccf8cd948-76htb"
Sep 09 16:13:06.991483 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:06.991301    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/de20207a-fe5a-4afd-854b-3af462d9572f-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-pxwsx\" (UID: \"de20207a-fe5a-4afd-854b-3af462d9572f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-pxwsx"
Sep 09 16:13:07.037268 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:07.037237    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-pxwsx"
Sep 09 16:13:07.045118 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:07.045095    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7ccf8cd948-76htb"
Sep 09 16:13:07.307335 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:07.307252    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-pxwsx"]
Sep 09 16:13:07.328926 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:07.328904    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7ccf8cd948-76htb"]
Sep 09 16:13:07.331841 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:13:07.331813    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod071d80b2_9f57_45c3_980d_678dc98590f1.slice/crio-3839fb044d23e69b793a7084204008ebbee59e4509cddd5f7707722ff2fe7f00 WatchSource:0}: Error finding container 3839fb044d23e69b793a7084204008ebbee59e4509cddd5f7707722ff2fe7f00: Status 404 returned error can't find the container with id 3839fb044d23e69b793a7084204008ebbee59e4509cddd5f7707722ff2fe7f00
Sep 09 16:13:07.780114 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:07.780072    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-pxwsx" event={"ID":"de20207a-fe5a-4afd-854b-3af462d9572f","Type":"ContainerStarted","Data":"141be4573e2257e9788a0678ea2fd6fc5c1f350db08e8f35df6587705af1b4be"}
Sep 09 16:13:07.781652 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:07.781627    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7ccf8cd948-76htb" event={"ID":"071d80b2-9f57-45c3-980d-678dc98590f1","Type":"ContainerStarted","Data":"1a5a9ec4f8fa7abafcd2fc27afcf0382ed5d1f3a1de2378c4eb1be7e94d886a1"}
Sep 09 16:13:07.781781 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:07.781671    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7ccf8cd948-76htb" event={"ID":"071d80b2-9f57-45c3-980d-678dc98590f1","Type":"ContainerStarted","Data":"3839fb044d23e69b793a7084204008ebbee59e4509cddd5f7707722ff2fe7f00"}
Sep 09 16:13:07.781832 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:07.781780    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7ccf8cd948-76htb"
Sep 09 16:13:07.799743 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:07.799665    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7ccf8cd948-76htb" podStartSLOduration=16.799651377 podStartE2EDuration="16.799651377s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 16:12:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 16:13:07.798481124 +0000 UTC m=+263.216331123" watchObservedRunningTime="2026-09-09 16:13:07.799651377 +0000 UTC m=+263.217501410"
Sep 09 16:13:09.791781 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:09.791741    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-pxwsx" event={"ID":"de20207a-fe5a-4afd-854b-3af462d9572f","Type":"ContainerStarted","Data":"1739fb307a3788476022a538357f6eb177c2507b437c77f66d79ef0a26ba6072"}
Sep 09 16:13:09.792177 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:09.791793    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-pxwsx" event={"ID":"de20207a-fe5a-4afd-854b-3af462d9572f","Type":"ContainerStarted","Data":"33d16fc0747d5f5e84265743a2708ce9785970fdbb51ce974083a1bb0a4d3863"}
Sep 09 16:13:09.809741 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:09.809683    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-pxwsx" podStartSLOduration=16.87026322 podStartE2EDuration="18.809665749s" totalImagesPullingTime="1.939402529s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 16:12:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 16:13:09.808987122 +0000 UTC m=+265.226837121" watchObservedRunningTime="2026-09-09 16:13:09.809665749 +0000 UTC m=+265.227515749"
Sep 09 16:13:13.043050 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:13.043015    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cade2714-b3c7-4de0-841b-b2f3bc582e6c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lg6v8\" (UID: \"cade2714-b3c7-4de0-841b-b2f3bc582e6c\") " pod="openshift-insights/insights-runtime-extractor-lg6v8"
Sep 09 16:13:13.043050 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:13.043048    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cade2714-b3c7-4de0-841b-b2f3bc582e6c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lg6v8\" (UID: \"cade2714-b3c7-4de0-841b-b2f3bc582e6c\") " pod="openshift-insights/insights-runtime-extractor-lg6v8"
Sep 09 16:13:13.043519 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:13.043503    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cade2714-b3c7-4de0-841b-b2f3bc582e6c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lg6v8\" (UID: \"cade2714-b3c7-4de0-841b-b2f3bc582e6c\") " pod="openshift-insights/insights-runtime-extractor-lg6v8"
Sep 09 16:13:13.045434 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:13.045414    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cade2714-b3c7-4de0-841b-b2f3bc582e6c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lg6v8\" (UID: \"cade2714-b3c7-4de0-841b-b2f3bc582e6c\") " pod="openshift-insights/insights-runtime-extractor-lg6v8"
Sep 09 16:13:13.102612 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:13.102574    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lg6v8"
Sep 09 16:13:13.375910 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:13.375869    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lg6v8"]
Sep 09 16:13:13.378824 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:13:13.378796    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcade2714_b3c7_4de0_841b_b2f3bc582e6c.slice/crio-87cf214ae37af653f72f7ed2a45d440ab690a41a1d8eaf394a7a18f9c000d334 WatchSource:0}: Error finding container 87cf214ae37af653f72f7ed2a45d440ab690a41a1d8eaf394a7a18f9c000d334: Status 404 returned error can't find the container with id 87cf214ae37af653f72f7ed2a45d440ab690a41a1d8eaf394a7a18f9c000d334
Sep 09 16:13:13.811839 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:13.811753    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lg6v8" event={"ID":"cade2714-b3c7-4de0-841b-b2f3bc582e6c","Type":"ContainerStarted","Data":"847bcd57507d037271cd805bbbd2f7a7579c32efc5a0b9a4e989c4094909bab7"}
Sep 09 16:13:13.811839 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:13.811831    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lg6v8" event={"ID":"cade2714-b3c7-4de0-841b-b2f3bc582e6c","Type":"ContainerStarted","Data":"87cf214ae37af653f72f7ed2a45d440ab690a41a1d8eaf394a7a18f9c000d334"}
Sep 09 16:13:14.816799 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:14.816768    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lg6v8" event={"ID":"cade2714-b3c7-4de0-841b-b2f3bc582e6c","Type":"ContainerStarted","Data":"f3f3221055e29ae831645f596ff70579d6b151b6c2fa2efc2b80e405c4b59b15"}
Sep 09 16:13:15.821410 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:15.821382    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lg6v8" event={"ID":"cade2714-b3c7-4de0-841b-b2f3bc582e6c","Type":"ContainerStarted","Data":"3a69146eeaf0bc541026612acafe8f60fd65ed30a3c155a8f6efb5dd45d84779"}
Sep 09 16:13:15.840846 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:15.840804    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-lg6v8" podStartSLOduration=16.600621355 podStartE2EDuration="18.840789637s" totalImagesPullingTime="2.240168282s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 16:12:57 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 16:13:15.839779746 +0000 UTC m=+271.257629746" watchObservedRunningTime="2026-09-09 16:13:15.840789637 +0000 UTC m=+271.258639636"
Sep 09 16:13:16.030439 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:16.030355    2567 scope.go:117] "RemoveContainer" containerID="b8b9072853592f000cfe83bd43871e328b5efa114b2a093d70a6a1a6d7a32f33"
Sep 09 16:13:16.826170 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:16.826140    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-xwlzs_16ecfb4d-7944-465e-a38e-d8c5bda6b1ac/console-operator/2.log"
Sep 09 16:13:16.826653 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:16.826602    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-xwlzs_16ecfb4d-7944-465e-a38e-d8c5bda6b1ac/console-operator/1.log"
Sep 09 16:13:16.826712 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:16.826643    2567 generic.go:356] "Generic (PLEG): container finished" podID="16ecfb4d-7944-465e-a38e-d8c5bda6b1ac" containerID="40476ca069f118fa8524f1f7f48aafd5b8e5255ccb4228a14dbdf85ce792a657" exitCode=255
Sep 09 16:13:16.826764 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:16.826724    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-xwlzs" event={"ID":"16ecfb4d-7944-465e-a38e-d8c5bda6b1ac","Type":"ContainerDied","Data":"40476ca069f118fa8524f1f7f48aafd5b8e5255ccb4228a14dbdf85ce792a657"}
Sep 09 16:13:16.826812 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:16.826786    2567 scope.go:117] "RemoveContainer" containerID="b8b9072853592f000cfe83bd43871e328b5efa114b2a093d70a6a1a6d7a32f33"
Sep 09 16:13:16.827218 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:16.827198    2567 scope.go:117] "RemoveContainer" containerID="40476ca069f118fa8524f1f7f48aafd5b8e5255ccb4228a14dbdf85ce792a657"
Sep 09 16:13:16.827387 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:13:16.827366    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85b454874b-xwlzs_openshift-console-operator(16ecfb4d-7944-465e-a38e-d8c5bda6b1ac)\"" pod="openshift-console-operator/console-operator-85b454874b-xwlzs" podUID="16ecfb4d-7944-465e-a38e-d8c5bda6b1ac"
Sep 09 16:13:17.831038 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:17.831005    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-xwlzs_16ecfb4d-7944-465e-a38e-d8c5bda6b1ac/console-operator/2.log"
Sep 09 16:13:20.787311 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:20.787276    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7ccf8cd948-76htb"]
Sep 09 16:13:21.450978 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:21.450933    2567 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-xwlzs"
Sep 09 16:13:21.451159 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:21.450997    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-xwlzs"
Sep 09 16:13:21.451319 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:21.451306    2567 scope.go:117] "RemoveContainer" containerID="40476ca069f118fa8524f1f7f48aafd5b8e5255ccb4228a14dbdf85ce792a657"
Sep 09 16:13:21.451513 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:13:21.451494    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85b454874b-xwlzs_openshift-console-operator(16ecfb4d-7944-465e-a38e-d8c5bda6b1ac)\"" pod="openshift-console-operator/console-operator-85b454874b-xwlzs" podUID="16ecfb4d-7944-465e-a38e-d8c5bda6b1ac"
Sep 09 16:13:22.921117 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:22.921075    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ae67a1e-f25a-4218-a9ce-d4614dd65d43-service-ca-bundle\") pod \"router-default-7d7756ccd6-mfpk8\" (UID: \"5ae67a1e-f25a-4218-a9ce-d4614dd65d43\") " pod="openshift-ingress/router-default-7d7756ccd6-mfpk8"
Sep 09 16:13:22.921535 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:22.921143    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ae67a1e-f25a-4218-a9ce-d4614dd65d43-metrics-certs\") pod \"router-default-7d7756ccd6-mfpk8\" (UID: \"5ae67a1e-f25a-4218-a9ce-d4614dd65d43\") " pod="openshift-ingress/router-default-7d7756ccd6-mfpk8"
Sep 09 16:13:22.921737 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:22.921715    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ae67a1e-f25a-4218-a9ce-d4614dd65d43-service-ca-bundle\") pod \"router-default-7d7756ccd6-mfpk8\" (UID: \"5ae67a1e-f25a-4218-a9ce-d4614dd65d43\") " pod="openshift-ingress/router-default-7d7756ccd6-mfpk8"
Sep 09 16:13:22.923593 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:22.923575    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ae67a1e-f25a-4218-a9ce-d4614dd65d43-metrics-certs\") pod \"router-default-7d7756ccd6-mfpk8\" (UID: \"5ae67a1e-f25a-4218-a9ce-d4614dd65d43\") " pod="openshift-ingress/router-default-7d7756ccd6-mfpk8"
Sep 09 16:13:23.140735 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:23.140695    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7d7756ccd6-mfpk8"
Sep 09 16:13:23.411560 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:23.411530    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7d7756ccd6-mfpk8"]
Sep 09 16:13:23.414753 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:13:23.414722    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ae67a1e_f25a_4218_a9ce_d4614dd65d43.slice/crio-c5ceea97cb3319495a5b344dbef448f41121cb382c1908d605a7bfff4781e028 WatchSource:0}: Error finding container c5ceea97cb3319495a5b344dbef448f41121cb382c1908d605a7bfff4781e028: Status 404 returned error can't find the container with id c5ceea97cb3319495a5b344dbef448f41121cb382c1908d605a7bfff4781e028
Sep 09 16:13:23.852791 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:23.852756    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7d7756ccd6-mfpk8" event={"ID":"5ae67a1e-f25a-4218-a9ce-d4614dd65d43","Type":"ContainerStarted","Data":"2c8c4b2d29cd1bd10ef86c40ca016e40488389186ac2340baffb1cd83ab3d08f"}
Sep 09 16:13:23.852940 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:23.852803    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7d7756ccd6-mfpk8" event={"ID":"5ae67a1e-f25a-4218-a9ce-d4614dd65d43","Type":"ContainerStarted","Data":"c5ceea97cb3319495a5b344dbef448f41121cb382c1908d605a7bfff4781e028"}
Sep 09 16:13:23.871994 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:23.871943    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-7d7756ccd6-mfpk8" podStartSLOduration=32.87192647 podStartE2EDuration="32.87192647s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 16:12:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 16:13:23.870950354 +0000 UTC m=+279.288800334" watchObservedRunningTime="2026-09-09 16:13:23.87192647 +0000 UTC m=+279.289776468"
Sep 09 16:13:24.141435 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:24.141347    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-7d7756ccd6-mfpk8"
Sep 09 16:13:24.144001 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:24.143979    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-7d7756ccd6-mfpk8"
Sep 09 16:13:24.855562 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:24.855530    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-7d7756ccd6-mfpk8"
Sep 09 16:13:24.856750 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:24.856728    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-7d7756ccd6-mfpk8"
Sep 09 16:13:25.498615 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:13:25.498572    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-frbps" podUID="8f8cbff3-9dfd-4406-9c22-260bd366d946"
Sep 09 16:13:25.858493 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:25.858445    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-frbps"
Sep 09 16:13:25.883491 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:25.883441    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-m5hq5"]
Sep 09 16:13:25.886671 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:25.886655    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-m5hq5"
Sep 09 16:13:25.888980 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:25.888955    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 09 16:13:25.889092 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:25.889039    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-vmngh\""
Sep 09 16:13:25.896421 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:25.896398    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-m5hq5"]
Sep 09 16:13:25.942373 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:25.942326    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d6c84114-6c4c-4a95-95c3-057ec1ddf0e3-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-m5hq5\" (UID: \"d6c84114-6c4c-4a95-95c3-057ec1ddf0e3\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-m5hq5"
Sep 09 16:13:26.043811 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:26.043775    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d6c84114-6c4c-4a95-95c3-057ec1ddf0e3-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-m5hq5\" (UID: \"d6c84114-6c4c-4a95-95c3-057ec1ddf0e3\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-m5hq5"
Sep 09 16:13:26.046345 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:26.046317    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d6c84114-6c4c-4a95-95c3-057ec1ddf0e3-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-m5hq5\" (UID: \"d6c84114-6c4c-4a95-95c3-057ec1ddf0e3\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-m5hq5"
Sep 09 16:13:26.195733 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:26.195617    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-m5hq5"
Sep 09 16:13:26.463789 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:26.463716    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-m5hq5"]
Sep 09 16:13:26.466713 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:13:26.466682    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6c84114_6c4c_4a95_95c3_057ec1ddf0e3.slice/crio-b5423dcae457d89136a8dccc6b7c73dd60160e2cb45093ac82bd98dff6a07c5b WatchSource:0}: Error finding container b5423dcae457d89136a8dccc6b7c73dd60160e2cb45093ac82bd98dff6a07c5b: Status 404 returned error can't find the container with id b5423dcae457d89136a8dccc6b7c73dd60160e2cb45093ac82bd98dff6a07c5b
Sep 09 16:13:26.863055 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:26.863018    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-m5hq5" event={"ID":"d6c84114-6c4c-4a95-95c3-057ec1ddf0e3","Type":"ContainerStarted","Data":"b5423dcae457d89136a8dccc6b7c73dd60160e2cb45093ac82bd98dff6a07c5b"}
Sep 09 16:13:29.368584 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:29.368541    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8f8cbff3-9dfd-4406-9c22-260bd366d946-metrics-tls\") pod \"dns-default-frbps\" (UID: \"8f8cbff3-9dfd-4406-9c22-260bd366d946\") " pod="openshift-dns/dns-default-frbps"
Sep 09 16:13:29.370939 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:29.370919    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8f8cbff3-9dfd-4406-9c22-260bd366d946-metrics-tls\") pod \"dns-default-frbps\" (UID: \"8f8cbff3-9dfd-4406-9c22-260bd366d946\") " pod="openshift-dns/dns-default-frbps"
Sep 09 16:13:29.461628 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:29.461602    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-pgn4p\""
Sep 09 16:13:29.469551 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:29.469529    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-frbps"
Sep 09 16:13:29.749152 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:29.749070    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-frbps"]
Sep 09 16:13:29.757390 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:13:29.757365    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f8cbff3_9dfd_4406_9c22_260bd366d946.slice/crio-8473ebdfb98b1690608909ce39dfd2635530db7010d8e058ea38801310374498 WatchSource:0}: Error finding container 8473ebdfb98b1690608909ce39dfd2635530db7010d8e058ea38801310374498: Status 404 returned error can't find the container with id 8473ebdfb98b1690608909ce39dfd2635530db7010d8e058ea38801310374498
Sep 09 16:13:29.875249 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:29.875212    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-frbps" event={"ID":"8f8cbff3-9dfd-4406-9c22-260bd366d946","Type":"ContainerStarted","Data":"8473ebdfb98b1690608909ce39dfd2635530db7010d8e058ea38801310374498"}
Sep 09 16:13:30.795814 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:30.795705    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7ccf8cd948-76htb"
Sep 09 16:13:30.880985 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:30.880941    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-m5hq5" event={"ID":"d6c84114-6c4c-4a95-95c3-057ec1ddf0e3","Type":"ContainerStarted","Data":"224c346c39ff450fd82199b097461deb5e893f1cc0882c73a0a9fbfb6f8756e4"}
Sep 09 16:13:30.881169 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:30.881140    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-m5hq5"
Sep 09 16:13:30.886532 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:30.886505    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-m5hq5"
Sep 09 16:13:30.896856 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:30.896732    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-m5hq5" podStartSLOduration=1.989215304 podStartE2EDuration="5.896717696s" totalImagesPullingTime="3.907502392s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 16:13:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 16:13:30.896241568 +0000 UTC m=+286.314091578" watchObservedRunningTime="2026-09-09 16:13:30.896717696 +0000 UTC m=+286.314567695"
Sep 09 16:13:31.943060 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:31.942699    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-8tttv"]
Sep 09 16:13:31.947806 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:31.947264    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-8tttv"
Sep 09 16:13:31.950169 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:31.950147    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 09 16:13:31.951053 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:31.951030    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 09 16:13:31.951217 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:31.951087    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 09 16:13:31.951217 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:31.951030    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 09 16:13:31.951217 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:31.951134    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-pt2jj\""
Sep 09 16:13:31.951217 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:31.951096    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 09 16:13:31.953654 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:31.953583    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-8tttv"]
Sep 09 16:13:31.988195 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:31.988157    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/10c36f54-5efc-4f1b-aba7-7e4e3116aea7-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-8tttv\" (UID: \"10c36f54-5efc-4f1b-aba7-7e4e3116aea7\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-8tttv"
Sep 09 16:13:31.988323 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:31.988268    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/10c36f54-5efc-4f1b-aba7-7e4e3116aea7-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-8tttv\" (UID: \"10c36f54-5efc-4f1b-aba7-7e4e3116aea7\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-8tttv"
Sep 09 16:13:31.988400 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:31.988326    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/10c36f54-5efc-4f1b-aba7-7e4e3116aea7-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-8tttv\" (UID: \"10c36f54-5efc-4f1b-aba7-7e4e3116aea7\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-8tttv"
Sep 09 16:13:31.988400 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:31.988357    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t25dk\" (UniqueName: \"kubernetes.io/projected/10c36f54-5efc-4f1b-aba7-7e4e3116aea7-kube-api-access-t25dk\") pod \"prometheus-operator-6f4946db56-8tttv\" (UID: \"10c36f54-5efc-4f1b-aba7-7e4e3116aea7\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-8tttv"
Sep 09 16:13:32.089793 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:32.089685    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/10c36f54-5efc-4f1b-aba7-7e4e3116aea7-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-8tttv\" (UID: \"10c36f54-5efc-4f1b-aba7-7e4e3116aea7\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-8tttv"
Sep 09 16:13:32.089793 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:32.089740    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t25dk\" (UniqueName: \"kubernetes.io/projected/10c36f54-5efc-4f1b-aba7-7e4e3116aea7-kube-api-access-t25dk\") pod \"prometheus-operator-6f4946db56-8tttv\" (UID: \"10c36f54-5efc-4f1b-aba7-7e4e3116aea7\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-8tttv"
Sep 09 16:13:32.090000 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:32.089879    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/10c36f54-5efc-4f1b-aba7-7e4e3116aea7-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-8tttv\" (UID: \"10c36f54-5efc-4f1b-aba7-7e4e3116aea7\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-8tttv"
Sep 09 16:13:32.090000 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:32.089947    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/10c36f54-5efc-4f1b-aba7-7e4e3116aea7-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-8tttv\" (UID: \"10c36f54-5efc-4f1b-aba7-7e4e3116aea7\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-8tttv"
Sep 09 16:13:32.090121 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:13:32.090079    2567 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Sep 09 16:13:32.090197 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:13:32.090135    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10c36f54-5efc-4f1b-aba7-7e4e3116aea7-prometheus-operator-tls podName:10c36f54-5efc-4f1b-aba7-7e4e3116aea7 nodeName:}" failed. No retries permitted until 2026-09-09 16:13:32.590118252 +0000 UTC m=+288.007968235 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/10c36f54-5efc-4f1b-aba7-7e4e3116aea7-prometheus-operator-tls") pod "prometheus-operator-6f4946db56-8tttv" (UID: "10c36f54-5efc-4f1b-aba7-7e4e3116aea7") : secret "prometheus-operator-tls" not found
Sep 09 16:13:32.090441 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:32.090419    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/10c36f54-5efc-4f1b-aba7-7e4e3116aea7-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-8tttv\" (UID: \"10c36f54-5efc-4f1b-aba7-7e4e3116aea7\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-8tttv"
Sep 09 16:13:32.092415 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:32.092394    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/10c36f54-5efc-4f1b-aba7-7e4e3116aea7-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-8tttv\" (UID: \"10c36f54-5efc-4f1b-aba7-7e4e3116aea7\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-8tttv"
Sep 09 16:13:32.098955 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:32.098933    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t25dk\" (UniqueName: \"kubernetes.io/projected/10c36f54-5efc-4f1b-aba7-7e4e3116aea7-kube-api-access-t25dk\") pod \"prometheus-operator-6f4946db56-8tttv\" (UID: \"10c36f54-5efc-4f1b-aba7-7e4e3116aea7\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-8tttv"
Sep 09 16:13:32.592784 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:32.592746    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/10c36f54-5efc-4f1b-aba7-7e4e3116aea7-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-8tttv\" (UID: \"10c36f54-5efc-4f1b-aba7-7e4e3116aea7\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-8tttv"
Sep 09 16:13:32.595262 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:32.595237    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/10c36f54-5efc-4f1b-aba7-7e4e3116aea7-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-8tttv\" (UID: \"10c36f54-5efc-4f1b-aba7-7e4e3116aea7\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-8tttv"
Sep 09 16:13:32.861207 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:32.861098    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-8tttv"
Sep 09 16:13:32.889836 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:32.889794    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-frbps" event={"ID":"8f8cbff3-9dfd-4406-9c22-260bd366d946","Type":"ContainerStarted","Data":"a225c54e299fb997096a93c2e745ddd3081537f49db95be9f33e7d5848b2e2cb"}
Sep 09 16:13:32.889959 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:32.889859    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-frbps" event={"ID":"8f8cbff3-9dfd-4406-9c22-260bd366d946","Type":"ContainerStarted","Data":"920b4c26329c6053577cafb4cb9111dfb61ddd4c29edef465ff555906704eb16"}
Sep 09 16:13:32.913450 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:32.912255    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-frbps" podStartSLOduration=251.842784287 podStartE2EDuration="4m13.912233849s" totalImagesPullingTime="2.069449562s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 16:09:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 16:13:32.911042244 +0000 UTC m=+288.328892243" watchObservedRunningTime="2026-09-09 16:13:32.912233849 +0000 UTC m=+288.330083846"
Sep 09 16:13:33.134758 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:33.134679    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-8tttv"]
Sep 09 16:13:33.137373 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:13:33.137348    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10c36f54_5efc_4f1b_aba7_7e4e3116aea7.slice/crio-a97d3e6472d1ac03da5c4d2a7e88fc3232d576fa2fd8dab093eb4cbe90ee59d5 WatchSource:0}: Error finding container a97d3e6472d1ac03da5c4d2a7e88fc3232d576fa2fd8dab093eb4cbe90ee59d5: Status 404 returned error can't find the container with id a97d3e6472d1ac03da5c4d2a7e88fc3232d576fa2fd8dab093eb4cbe90ee59d5
Sep 09 16:13:33.897103 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:33.897061    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-8tttv" event={"ID":"10c36f54-5efc-4f1b-aba7-7e4e3116aea7","Type":"ContainerStarted","Data":"a97d3e6472d1ac03da5c4d2a7e88fc3232d576fa2fd8dab093eb4cbe90ee59d5"}
Sep 09 16:13:33.897318 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:33.897231    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-frbps"
Sep 09 16:13:34.030037 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:34.030007    2567 scope.go:117] "RemoveContainer" containerID="40476ca069f118fa8524f1f7f48aafd5b8e5255ccb4228a14dbdf85ce792a657"
Sep 09 16:13:34.030218 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:13:34.030198    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85b454874b-xwlzs_openshift-console-operator(16ecfb4d-7944-465e-a38e-d8c5bda6b1ac)\"" pod="openshift-console-operator/console-operator-85b454874b-xwlzs" podUID="16ecfb4d-7944-465e-a38e-d8c5bda6b1ac"
Sep 09 16:13:34.908815 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:34.908774    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-8tttv" event={"ID":"10c36f54-5efc-4f1b-aba7-7e4e3116aea7","Type":"ContainerStarted","Data":"cf3893c268d4548d53cc65aadf1ab112ecf18d21187b42a4b12b451b5b2d0a5b"}
Sep 09 16:13:34.909312 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:34.908874    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-8tttv" event={"ID":"10c36f54-5efc-4f1b-aba7-7e4e3116aea7","Type":"ContainerStarted","Data":"30ce78d43fc615160e96818268fe48001a2c52f7b28d0c5b693b3d793ef2e8e9"}
Sep 09 16:13:34.925274 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:34.925226    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-6f4946db56-8tttv" podStartSLOduration=2.321523979 podStartE2EDuration="3.925208723s" totalImagesPullingTime="1.603684744s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 16:13:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 16:13:34.92418446 +0000 UTC m=+290.342034460" watchObservedRunningTime="2026-09-09 16:13:34.925208723 +0000 UTC m=+290.343058721"
Sep 09 16:13:37.309853 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:37.309820    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-6b5vr"]
Sep 09 16:13:37.313007 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:37.312987    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-6b5vr"
Sep 09 16:13:37.317642 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:37.317619    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 09 16:13:37.318345 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:37.318328    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-vfh4m\""
Sep 09 16:13:37.318570 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:37.318551    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 09 16:13:37.324821 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:37.324800    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-6b5vr"]
Sep 09 16:13:37.361989 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:37.361957    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-xwz7w"]
Sep 09 16:13:37.365133 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:37.365116    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-xwz7w"
Sep 09 16:13:37.368050 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:37.368022    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 09 16:13:37.368169 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:37.368098    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 09 16:13:37.368231 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:37.368191    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-9hbdg\""
Sep 09 16:13:37.368327 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:37.368310    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 09 16:13:37.428503 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:37.428441    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/32e8c12d-3457-493b-b124-519623fbe249-root\") pod \"node-exporter-xwz7w\" (UID: \"32e8c12d-3457-493b-b124-519623fbe249\") " pod="openshift-monitoring/node-exporter-xwz7w"
Sep 09 16:13:37.428661 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:37.428512    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/32e8c12d-3457-493b-b124-519623fbe249-sys\") pod \"node-exporter-xwz7w\" (UID: \"32e8c12d-3457-493b-b124-519623fbe249\") " pod="openshift-monitoring/node-exporter-xwz7w"
Sep 09 16:13:37.428661 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:37.428532    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/32e8c12d-3457-493b-b124-519623fbe249-node-exporter-textfile\") pod \"node-exporter-xwz7w\" (UID: \"32e8c12d-3457-493b-b124-519623fbe249\") " pod="openshift-monitoring/node-exporter-xwz7w"
Sep 09 16:13:37.428661 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:37.428549    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/32e8c12d-3457-493b-b124-519623fbe249-node-exporter-wtmp\") pod \"node-exporter-xwz7w\" (UID: \"32e8c12d-3457-493b-b124-519623fbe249\") " pod="openshift-monitoring/node-exporter-xwz7w"
Sep 09 16:13:37.428661 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:37.428563    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/18af883a-dc08-484a-984a-6db038a684ed-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-6b5vr\" (UID: \"18af883a-dc08-484a-984a-6db038a684ed\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-6b5vr"
Sep 09 16:13:37.428661 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:37.428591    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/18af883a-dc08-484a-984a-6db038a684ed-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-6b5vr\" (UID: \"18af883a-dc08-484a-984a-6db038a684ed\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-6b5vr"
Sep 09 16:13:37.428939 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:37.428700    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/32e8c12d-3457-493b-b124-519623fbe249-node-exporter-accelerators-collector-config\") pod \"node-exporter-xwz7w\" (UID: \"32e8c12d-3457-493b-b124-519623fbe249\") " pod="openshift-monitoring/node-exporter-xwz7w"
Sep 09 16:13:37.428939 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:37.428754    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqrzz\" (UniqueName: \"kubernetes.io/projected/32e8c12d-3457-493b-b124-519623fbe249-kube-api-access-qqrzz\") pod \"node-exporter-xwz7w\" (UID: \"32e8c12d-3457-493b-b124-519623fbe249\") " pod="openshift-monitoring/node-exporter-xwz7w"
Sep 09 16:13:37.428939 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:37.428789    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/32e8c12d-3457-493b-b124-519623fbe249-metrics-client-ca\") pod \"node-exporter-xwz7w\" (UID: \"32e8c12d-3457-493b-b124-519623fbe249\") " pod="openshift-monitoring/node-exporter-xwz7w"
Sep 09 16:13:37.428939 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:37.428872    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fk5rr\" (UniqueName: \"kubernetes.io/projected/18af883a-dc08-484a-984a-6db038a684ed-kube-api-access-fk5rr\") pod \"openshift-state-metrics-657fd6c54-6b5vr\" (UID: \"18af883a-dc08-484a-984a-6db038a684ed\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-6b5vr"
Sep 09 16:13:37.428939 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:37.428913    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/32e8c12d-3457-493b-b124-519623fbe249-node-exporter-tls\") pod \"node-exporter-xwz7w\" (UID: \"32e8c12d-3457-493b-b124-519623fbe249\") " pod="openshift-monitoring/node-exporter-xwz7w"
Sep 09 16:13:37.429121 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:37.428940    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/32e8c12d-3457-493b-b124-519623fbe249-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xwz7w\" (UID: \"32e8c12d-3457-493b-b124-519623fbe249\") " pod="openshift-monitoring/node-exporter-xwz7w"
Sep 09 16:13:37.429121 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:37.428966    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/18af883a-dc08-484a-984a-6db038a684ed-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-6b5vr\" (UID: \"18af883a-dc08-484a-984a-6db038a684ed\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-6b5vr"
Sep 09 16:13:37.529965 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:37.529933    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/32e8c12d-3457-493b-b124-519623fbe249-metrics-client-ca\") pod \"node-exporter-xwz7w\" (UID: \"32e8c12d-3457-493b-b124-519623fbe249\") " pod="openshift-monitoring/node-exporter-xwz7w"
Sep 09 16:13:37.530129 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:37.529983    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fk5rr\" (UniqueName: \"kubernetes.io/projected/18af883a-dc08-484a-984a-6db038a684ed-kube-api-access-fk5rr\") pod \"openshift-state-metrics-657fd6c54-6b5vr\" (UID: \"18af883a-dc08-484a-984a-6db038a684ed\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-6b5vr"
Sep 09 16:13:37.530129 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:37.530015    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/32e8c12d-3457-493b-b124-519623fbe249-node-exporter-tls\") pod \"node-exporter-xwz7w\" (UID: \"32e8c12d-3457-493b-b124-519623fbe249\") " pod="openshift-monitoring/node-exporter-xwz7w"
Sep 09 16:13:37.530129 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:37.530033    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/32e8c12d-3457-493b-b124-519623fbe249-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xwz7w\" (UID: \"32e8c12d-3457-493b-b124-519623fbe249\") " pod="openshift-monitoring/node-exporter-xwz7w"
Sep 09 16:13:37.530264 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:13:37.530139    2567 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 09 16:13:37.530264 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:13:37.530202    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32e8c12d-3457-493b-b124-519623fbe249-node-exporter-tls podName:32e8c12d-3457-493b-b124-519623fbe249 nodeName:}" failed. No retries permitted until 2026-09-09 16:13:38.030183181 +0000 UTC m=+293.448033158 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/32e8c12d-3457-493b-b124-519623fbe249-node-exporter-tls") pod "node-exporter-xwz7w" (UID: "32e8c12d-3457-493b-b124-519623fbe249") : secret "node-exporter-tls" not found
Sep 09 16:13:37.530384 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:37.530277    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/18af883a-dc08-484a-984a-6db038a684ed-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-6b5vr\" (UID: \"18af883a-dc08-484a-984a-6db038a684ed\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-6b5vr"
Sep 09 16:13:37.530384 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:37.530334    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/32e8c12d-3457-493b-b124-519623fbe249-root\") pod \"node-exporter-xwz7w\" (UID: \"32e8c12d-3457-493b-b124-519623fbe249\") " pod="openshift-monitoring/node-exporter-xwz7w"
Sep 09 16:13:37.530384 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:37.530373    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/32e8c12d-3457-493b-b124-519623fbe249-sys\") pod \"node-exporter-xwz7w\" (UID: \"32e8c12d-3457-493b-b124-519623fbe249\") " pod="openshift-monitoring/node-exporter-xwz7w"
Sep 09 16:13:37.530551 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:37.530473    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/32e8c12d-3457-493b-b124-519623fbe249-root\") pod \"node-exporter-xwz7w\" (UID: \"32e8c12d-3457-493b-b124-519623fbe249\") " pod="openshift-monitoring/node-exporter-xwz7w"
Sep 09 16:13:37.530551 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:37.530482    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/32e8c12d-3457-493b-b124-519623fbe249-sys\") pod \"node-exporter-xwz7w\" (UID: \"32e8c12d-3457-493b-b124-519623fbe249\") " pod="openshift-monitoring/node-exporter-xwz7w"
Sep 09 16:13:37.530551 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:37.530403    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/32e8c12d-3457-493b-b124-519623fbe249-node-exporter-textfile\") pod \"node-exporter-xwz7w\" (UID: \"32e8c12d-3457-493b-b124-519623fbe249\") " pod="openshift-monitoring/node-exporter-xwz7w"
Sep 09 16:13:37.530702 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:37.530553    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/32e8c12d-3457-493b-b124-519623fbe249-node-exporter-wtmp\") pod \"node-exporter-xwz7w\" (UID: \"32e8c12d-3457-493b-b124-519623fbe249\") " pod="openshift-monitoring/node-exporter-xwz7w"
Sep 09 16:13:37.530702 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:37.530599    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/18af883a-dc08-484a-984a-6db038a684ed-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-6b5vr\" (UID: \"18af883a-dc08-484a-984a-6db038a684ed\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-6b5vr"
Sep 09 16:13:37.530805 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:37.530736    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/32e8c12d-3457-493b-b124-519623fbe249-metrics-client-ca\") pod \"node-exporter-xwz7w\" (UID: \"32e8c12d-3457-493b-b124-519623fbe249\") " pod="openshift-monitoring/node-exporter-xwz7w"
Sep 09 16:13:37.530805 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:37.530749    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/32e8c12d-3457-493b-b124-519623fbe249-node-exporter-wtmp\") pod \"node-exporter-xwz7w\" (UID: \"32e8c12d-3457-493b-b124-519623fbe249\") " pod="openshift-monitoring/node-exporter-xwz7w"
Sep 09 16:13:37.530805 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:37.530792    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/18af883a-dc08-484a-984a-6db038a684ed-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-6b5vr\" (UID: \"18af883a-dc08-484a-984a-6db038a684ed\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-6b5vr"
Sep 09 16:13:37.530951 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:37.530840    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/32e8c12d-3457-493b-b124-519623fbe249-node-exporter-textfile\") pod \"node-exporter-xwz7w\" (UID: \"32e8c12d-3457-493b-b124-519623fbe249\") " pod="openshift-monitoring/node-exporter-xwz7w"
Sep 09 16:13:37.531077 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:37.531054    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/32e8c12d-3457-493b-b124-519623fbe249-node-exporter-accelerators-collector-config\") pod \"node-exporter-xwz7w\" (UID: \"32e8c12d-3457-493b-b124-519623fbe249\") " pod="openshift-monitoring/node-exporter-xwz7w"
Sep 09 16:13:37.531212 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:37.531196    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqrzz\" (UniqueName: \"kubernetes.io/projected/32e8c12d-3457-493b-b124-519623fbe249-kube-api-access-qqrzz\") pod \"node-exporter-xwz7w\" (UID: \"32e8c12d-3457-493b-b124-519623fbe249\") " pod="openshift-monitoring/node-exporter-xwz7w"
Sep 09 16:13:37.531567 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:37.531519    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/18af883a-dc08-484a-984a-6db038a684ed-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-6b5vr\" (UID: \"18af883a-dc08-484a-984a-6db038a684ed\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-6b5vr"
Sep 09 16:13:37.532055 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:37.532032    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/32e8c12d-3457-493b-b124-519623fbe249-node-exporter-accelerators-collector-config\") pod \"node-exporter-xwz7w\" (UID: \"32e8c12d-3457-493b-b124-519623fbe249\") " pod="openshift-monitoring/node-exporter-xwz7w"
Sep 09 16:13:37.533284 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:37.533260    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/32e8c12d-3457-493b-b124-519623fbe249-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xwz7w\" (UID: \"32e8c12d-3457-493b-b124-519623fbe249\") " pod="openshift-monitoring/node-exporter-xwz7w"
Sep 09 16:13:37.533723 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:37.533700    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/18af883a-dc08-484a-984a-6db038a684ed-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-6b5vr\" (UID: \"18af883a-dc08-484a-984a-6db038a684ed\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-6b5vr"
Sep 09 16:13:37.533786 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:37.533711    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/18af883a-dc08-484a-984a-6db038a684ed-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-6b5vr\" (UID: \"18af883a-dc08-484a-984a-6db038a684ed\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-6b5vr"
Sep 09 16:13:37.538926 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:37.538899    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fk5rr\" (UniqueName: \"kubernetes.io/projected/18af883a-dc08-484a-984a-6db038a684ed-kube-api-access-fk5rr\") pod \"openshift-state-metrics-657fd6c54-6b5vr\" (UID: \"18af883a-dc08-484a-984a-6db038a684ed\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-6b5vr"
Sep 09 16:13:37.540090 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:37.540073    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqrzz\" (UniqueName: \"kubernetes.io/projected/32e8c12d-3457-493b-b124-519623fbe249-kube-api-access-qqrzz\") pod \"node-exporter-xwz7w\" (UID: \"32e8c12d-3457-493b-b124-519623fbe249\") " pod="openshift-monitoring/node-exporter-xwz7w"
Sep 09 16:13:37.622053 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:37.622011    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-6b5vr"
Sep 09 16:13:37.904954 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:37.904816    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-6b5vr"]
Sep 09 16:13:37.908733 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:13:37.908700    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18af883a_dc08_484a_984a_6db038a684ed.slice/crio-6f5a523a57aec1ca4c0c716f92155688424d2afe5205e7d395d6d9aa0e8104ba WatchSource:0}: Error finding container 6f5a523a57aec1ca4c0c716f92155688424d2afe5205e7d395d6d9aa0e8104ba: Status 404 returned error can't find the container with id 6f5a523a57aec1ca4c0c716f92155688424d2afe5205e7d395d6d9aa0e8104ba
Sep 09 16:13:37.919057 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:37.919032    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-6b5vr" event={"ID":"18af883a-dc08-484a-984a-6db038a684ed","Type":"ContainerStarted","Data":"6f5a523a57aec1ca4c0c716f92155688424d2afe5205e7d395d6d9aa0e8104ba"}
Sep 09 16:13:38.036397 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:38.036368    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/32e8c12d-3457-493b-b124-519623fbe249-node-exporter-tls\") pod \"node-exporter-xwz7w\" (UID: \"32e8c12d-3457-493b-b124-519623fbe249\") " pod="openshift-monitoring/node-exporter-xwz7w"
Sep 09 16:13:38.036555 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:13:38.036515    2567 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 09 16:13:38.036605 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:13:38.036567    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32e8c12d-3457-493b-b124-519623fbe249-node-exporter-tls podName:32e8c12d-3457-493b-b124-519623fbe249 nodeName:}" failed. No retries permitted until 2026-09-09 16:13:39.036548246 +0000 UTC m=+294.454398226 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/32e8c12d-3457-493b-b124-519623fbe249-node-exporter-tls") pod "node-exporter-xwz7w" (UID: "32e8c12d-3457-493b-b124-519623fbe249") : secret "node-exporter-tls" not found
Sep 09 16:13:38.923935 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:38.923898    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-6b5vr" event={"ID":"18af883a-dc08-484a-984a-6db038a684ed","Type":"ContainerStarted","Data":"2c35f646d6008fdb28393074da23b6d3701950b33430cebd7c5a83ed12b6e894"}
Sep 09 16:13:38.924312 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:38.923984    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-6b5vr" event={"ID":"18af883a-dc08-484a-984a-6db038a684ed","Type":"ContainerStarted","Data":"da7c2154b49274cee0c258701175fd329d5c5577a7ca97d262008dbbf66c74fe"}
Sep 09 16:13:39.046941 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:39.046901    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/32e8c12d-3457-493b-b124-519623fbe249-node-exporter-tls\") pod \"node-exporter-xwz7w\" (UID: \"32e8c12d-3457-493b-b124-519623fbe249\") " pod="openshift-monitoring/node-exporter-xwz7w"
Sep 09 16:13:39.049420 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:39.049395    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/32e8c12d-3457-493b-b124-519623fbe249-node-exporter-tls\") pod \"node-exporter-xwz7w\" (UID: \"32e8c12d-3457-493b-b124-519623fbe249\") " pod="openshift-monitoring/node-exporter-xwz7w"
Sep 09 16:13:39.173922 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:39.173875    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-xwz7w"
Sep 09 16:13:39.186279 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:13:39.186245    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32e8c12d_3457_493b_b124_519623fbe249.slice/crio-b0c8dc22d65efcafcf8019d12d03b675c39f94e036d008215ff60f0150b56ed8 WatchSource:0}: Error finding container b0c8dc22d65efcafcf8019d12d03b675c39f94e036d008215ff60f0150b56ed8: Status 404 returned error can't find the container with id b0c8dc22d65efcafcf8019d12d03b675c39f94e036d008215ff60f0150b56ed8
Sep 09 16:13:39.930139 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:39.930080    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-6b5vr" event={"ID":"18af883a-dc08-484a-984a-6db038a684ed","Type":"ContainerStarted","Data":"e4ee2faa3e8bb3f7318ac98cf1ddeacb1c0b03e4927a914aef49e1ca6ebb7cc8"}
Sep 09 16:13:39.931790 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:39.931760    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xwz7w" event={"ID":"32e8c12d-3457-493b-b124-519623fbe249","Type":"ContainerStarted","Data":"b0c8dc22d65efcafcf8019d12d03b675c39f94e036d008215ff60f0150b56ed8"}
Sep 09 16:13:39.950116 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:39.950065    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-6b5vr" podStartSLOduration=1.436737038 podStartE2EDuration="2.950047939s" totalImagesPullingTime="1.513310901s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 16:13:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 16:13:39.948782096 +0000 UTC m=+295.366632096" watchObservedRunningTime="2026-09-09 16:13:39.950047939 +0000 UTC m=+295.367897964"
Sep 09 16:13:40.936477 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:40.936428    2567 generic.go:356] "Generic (PLEG): container finished" podID="32e8c12d-3457-493b-b124-519623fbe249" containerID="86506e77a7ba4df4db8725938816a495bb2f3505a1c434d3774b9af0bf62cf7e" exitCode=0
Sep 09 16:13:40.936884 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:40.936521    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xwz7w" event={"ID":"32e8c12d-3457-493b-b124-519623fbe249","Type":"ContainerDied","Data":"86506e77a7ba4df4db8725938816a495bb2f3505a1c434d3774b9af0bf62cf7e"}
Sep 09 16:13:41.602037 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:41.601999    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-c47648b58-qxzjc"]
Sep 09 16:13:41.605152 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:41.605132    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-c47648b58-qxzjc"
Sep 09 16:13:41.607605 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:41.607580    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 09 16:13:41.608669 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:41.608643    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 09 16:13:41.608774 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:41.608644    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-bjgu8cba72uea\""
Sep 09 16:13:41.608774 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:41.608700    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 09 16:13:41.608774 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:41.608714    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 09 16:13:41.608774 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:41.608755    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-dhrm7\""
Sep 09 16:13:41.616366 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:41.616344    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-c47648b58-qxzjc"]
Sep 09 16:13:41.667853 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:41.667817    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/8be9e6b1-71f9-4055-914c-66b813d981ec-secret-metrics-server-client-certs\") pod \"metrics-server-c47648b58-qxzjc\" (UID: \"8be9e6b1-71f9-4055-914c-66b813d981ec\") " pod="openshift-monitoring/metrics-server-c47648b58-qxzjc"
Sep 09 16:13:41.667853 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:41.667853    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/8be9e6b1-71f9-4055-914c-66b813d981ec-secret-metrics-server-tls\") pod \"metrics-server-c47648b58-qxzjc\" (UID: \"8be9e6b1-71f9-4055-914c-66b813d981ec\") " pod="openshift-monitoring/metrics-server-c47648b58-qxzjc"
Sep 09 16:13:41.668056 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:41.667878    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8be9e6b1-71f9-4055-914c-66b813d981ec-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-c47648b58-qxzjc\" (UID: \"8be9e6b1-71f9-4055-914c-66b813d981ec\") " pod="openshift-monitoring/metrics-server-c47648b58-qxzjc"
Sep 09 16:13:41.668056 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:41.667957    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/8be9e6b1-71f9-4055-914c-66b813d981ec-audit-log\") pod \"metrics-server-c47648b58-qxzjc\" (UID: \"8be9e6b1-71f9-4055-914c-66b813d981ec\") " pod="openshift-monitoring/metrics-server-c47648b58-qxzjc"
Sep 09 16:13:41.668056 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:41.667990    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8be9e6b1-71f9-4055-914c-66b813d981ec-client-ca-bundle\") pod \"metrics-server-c47648b58-qxzjc\" (UID: \"8be9e6b1-71f9-4055-914c-66b813d981ec\") " pod="openshift-monitoring/metrics-server-c47648b58-qxzjc"
Sep 09 16:13:41.668056 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:41.668016    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/8be9e6b1-71f9-4055-914c-66b813d981ec-metrics-server-audit-profiles\") pod \"metrics-server-c47648b58-qxzjc\" (UID: \"8be9e6b1-71f9-4055-914c-66b813d981ec\") " pod="openshift-monitoring/metrics-server-c47648b58-qxzjc"
Sep 09 16:13:41.668056 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:41.668034    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhwgt\" (UniqueName: \"kubernetes.io/projected/8be9e6b1-71f9-4055-914c-66b813d981ec-kube-api-access-nhwgt\") pod \"metrics-server-c47648b58-qxzjc\" (UID: \"8be9e6b1-71f9-4055-914c-66b813d981ec\") " pod="openshift-monitoring/metrics-server-c47648b58-qxzjc"
Sep 09 16:13:41.769062 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:41.769026    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/8be9e6b1-71f9-4055-914c-66b813d981ec-secret-metrics-server-client-certs\") pod \"metrics-server-c47648b58-qxzjc\" (UID: \"8be9e6b1-71f9-4055-914c-66b813d981ec\") " pod="openshift-monitoring/metrics-server-c47648b58-qxzjc"
Sep 09 16:13:41.769062 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:41.769065    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/8be9e6b1-71f9-4055-914c-66b813d981ec-secret-metrics-server-tls\") pod \"metrics-server-c47648b58-qxzjc\" (UID: \"8be9e6b1-71f9-4055-914c-66b813d981ec\") " pod="openshift-monitoring/metrics-server-c47648b58-qxzjc"
Sep 09 16:13:41.769260 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:41.769096    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8be9e6b1-71f9-4055-914c-66b813d981ec-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-c47648b58-qxzjc\" (UID: \"8be9e6b1-71f9-4055-914c-66b813d981ec\") " pod="openshift-monitoring/metrics-server-c47648b58-qxzjc"
Sep 09 16:13:41.769260 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:41.769141    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/8be9e6b1-71f9-4055-914c-66b813d981ec-audit-log\") pod \"metrics-server-c47648b58-qxzjc\" (UID: \"8be9e6b1-71f9-4055-914c-66b813d981ec\") " pod="openshift-monitoring/metrics-server-c47648b58-qxzjc"
Sep 09 16:13:41.769260 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:41.769168    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8be9e6b1-71f9-4055-914c-66b813d981ec-client-ca-bundle\") pod \"metrics-server-c47648b58-qxzjc\" (UID: \"8be9e6b1-71f9-4055-914c-66b813d981ec\") " pod="openshift-monitoring/metrics-server-c47648b58-qxzjc"
Sep 09 16:13:41.769260 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:41.769196    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/8be9e6b1-71f9-4055-914c-66b813d981ec-metrics-server-audit-profiles\") pod \"metrics-server-c47648b58-qxzjc\" (UID: \"8be9e6b1-71f9-4055-914c-66b813d981ec\") " pod="openshift-monitoring/metrics-server-c47648b58-qxzjc"
Sep 09 16:13:41.769440 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:41.769266    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhwgt\" (UniqueName: \"kubernetes.io/projected/8be9e6b1-71f9-4055-914c-66b813d981ec-kube-api-access-nhwgt\") pod \"metrics-server-c47648b58-qxzjc\" (UID: \"8be9e6b1-71f9-4055-914c-66b813d981ec\") " pod="openshift-monitoring/metrics-server-c47648b58-qxzjc"
Sep 09 16:13:41.769815 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:41.769792    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/8be9e6b1-71f9-4055-914c-66b813d981ec-audit-log\") pod \"metrics-server-c47648b58-qxzjc\" (UID: \"8be9e6b1-71f9-4055-914c-66b813d981ec\") " pod="openshift-monitoring/metrics-server-c47648b58-qxzjc"
Sep 09 16:13:41.770289 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:41.770266    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/8be9e6b1-71f9-4055-914c-66b813d981ec-metrics-server-audit-profiles\") pod \"metrics-server-c47648b58-qxzjc\" (UID: \"8be9e6b1-71f9-4055-914c-66b813d981ec\") " pod="openshift-monitoring/metrics-server-c47648b58-qxzjc"
Sep 09 16:13:41.770391 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:41.770292    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8be9e6b1-71f9-4055-914c-66b813d981ec-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-c47648b58-qxzjc\" (UID: \"8be9e6b1-71f9-4055-914c-66b813d981ec\") " pod="openshift-monitoring/metrics-server-c47648b58-qxzjc"
Sep 09 16:13:41.772046 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:41.772026    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8be9e6b1-71f9-4055-914c-66b813d981ec-client-ca-bundle\") pod \"metrics-server-c47648b58-qxzjc\" (UID: \"8be9e6b1-71f9-4055-914c-66b813d981ec\") " pod="openshift-monitoring/metrics-server-c47648b58-qxzjc"
Sep 09 16:13:41.772256 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:41.772240    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/8be9e6b1-71f9-4055-914c-66b813d981ec-secret-metrics-server-tls\") pod \"metrics-server-c47648b58-qxzjc\" (UID: \"8be9e6b1-71f9-4055-914c-66b813d981ec\") " pod="openshift-monitoring/metrics-server-c47648b58-qxzjc"
Sep 09 16:13:41.772303 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:41.772262    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/8be9e6b1-71f9-4055-914c-66b813d981ec-secret-metrics-server-client-certs\") pod \"metrics-server-c47648b58-qxzjc\" (UID: \"8be9e6b1-71f9-4055-914c-66b813d981ec\") " pod="openshift-monitoring/metrics-server-c47648b58-qxzjc"
Sep 09 16:13:41.779270 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:41.779252    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhwgt\" (UniqueName: \"kubernetes.io/projected/8be9e6b1-71f9-4055-914c-66b813d981ec-kube-api-access-nhwgt\") pod \"metrics-server-c47648b58-qxzjc\" (UID: \"8be9e6b1-71f9-4055-914c-66b813d981ec\") " pod="openshift-monitoring/metrics-server-c47648b58-qxzjc"
Sep 09 16:13:41.914842 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:41.914757    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-c47648b58-qxzjc"
Sep 09 16:13:41.942979 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:41.942947    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xwz7w" event={"ID":"32e8c12d-3457-493b-b124-519623fbe249","Type":"ContainerStarted","Data":"d50a82365c865f0bff999313d89f9594f96c59ce2467099cf9cf71c0eafd5ed7"}
Sep 09 16:13:41.943336 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:41.943011    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xwz7w" event={"ID":"32e8c12d-3457-493b-b124-519623fbe249","Type":"ContainerStarted","Data":"e561067fd08fb23eed962a41f51bcb2021341d6291c9f297e2e89c8a8ffd3318"}
Sep 09 16:13:41.961718 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:41.961470    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-xwz7w" podStartSLOduration=3.766228225 podStartE2EDuration="4.961441275s" totalImagesPullingTime="1.12347198s" totalInitContainerRuntime="71.74107ms" isStatefulPod=true podCreationTimestamp="2026-09-09 16:13:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 16:13:41.96069922 +0000 UTC m=+297.378549221" watchObservedRunningTime="2026-09-09 16:13:41.961441275 +0000 UTC m=+297.379291311"
Sep 09 16:13:42.030658 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:13:42.030582    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-t29nl" podUID="73b8ff58-aa37-4fab-9632-9b0f5d409902"
Sep 09 16:13:42.189575 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:42.189499    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-c47648b58-qxzjc"]
Sep 09 16:13:42.192252 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:13:42.192228    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8be9e6b1_71f9_4055_914c_66b813d981ec.slice/crio-d12180124c7af1060afeaa5a7940fea5d10b7f3d24c2eb618048c3d0bd5301e2 WatchSource:0}: Error finding container d12180124c7af1060afeaa5a7940fea5d10b7f3d24c2eb618048c3d0bd5301e2: Status 404 returned error can't find the container with id d12180124c7af1060afeaa5a7940fea5d10b7f3d24c2eb618048c3d0bd5301e2
Sep 09 16:13:42.948342 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:42.948295    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-c47648b58-qxzjc" event={"ID":"8be9e6b1-71f9-4055-914c-66b813d981ec","Type":"ContainerStarted","Data":"d12180124c7af1060afeaa5a7940fea5d10b7f3d24c2eb618048c3d0bd5301e2"}
Sep 09 16:13:43.532060 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.532025    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 09 16:13:43.538581 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.538549    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:13:43.541410 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.541374    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 09 16:13:43.543790 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.542212    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 09 16:13:43.543790 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.542795    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-3hrttl78b20d8\""
Sep 09 16:13:43.544146 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.543886    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 09 16:13:43.544146 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.544033    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 09 16:13:43.544849 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.544410    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 09 16:13:43.544849 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.544567    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 09 16:13:43.544849 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.544605    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 09 16:13:43.544849 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.544645    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 09 16:13:43.544849 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.544704    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 09 16:13:43.545155 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.544930    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-gdp56\""
Sep 09 16:13:43.545155 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.545107    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 09 16:13:43.545369 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.545339    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 09 16:13:43.545854 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.545831    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 09 16:13:43.547129 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.547109    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 09 16:13:43.549489 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.549450    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 09 16:13:43.554916 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.554895    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 09 16:13:43.589659 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.589625    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/84e0c89a-ea89-4303-ab75-20cd7ddf0752-config\") pod \"prometheus-k8s-0\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:13:43.589834 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.589661    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/84e0c89a-ea89-4303-ab75-20cd7ddf0752-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:13:43.589834 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.589722    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/84e0c89a-ea89-4303-ab75-20cd7ddf0752-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:13:43.589834 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.589767    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/84e0c89a-ea89-4303-ab75-20cd7ddf0752-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:13:43.589834 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.589796    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/84e0c89a-ea89-4303-ab75-20cd7ddf0752-config-out\") pod \"prometheus-k8s-0\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:13:43.589834 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.589820    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84e0c89a-ea89-4303-ab75-20cd7ddf0752-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:13:43.590034 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.589849    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/84e0c89a-ea89-4303-ab75-20cd7ddf0752-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:13:43.590034 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.589918    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/84e0c89a-ea89-4303-ab75-20cd7ddf0752-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:13:43.590034 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.589951    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nw9s6\" (UniqueName: \"kubernetes.io/projected/84e0c89a-ea89-4303-ab75-20cd7ddf0752-kube-api-access-nw9s6\") pod \"prometheus-k8s-0\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:13:43.590034 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.590004    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84e0c89a-ea89-4303-ab75-20cd7ddf0752-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:13:43.590189 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.590032    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/84e0c89a-ea89-4303-ab75-20cd7ddf0752-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:13:43.590189 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.590056    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84e0c89a-ea89-4303-ab75-20cd7ddf0752-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:13:43.590189 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.590120    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/84e0c89a-ea89-4303-ab75-20cd7ddf0752-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:13:43.590189 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.590166    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/84e0c89a-ea89-4303-ab75-20cd7ddf0752-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:13:43.590325 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.590194    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/84e0c89a-ea89-4303-ab75-20cd7ddf0752-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:13:43.590325 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.590238    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/84e0c89a-ea89-4303-ab75-20cd7ddf0752-web-config\") pod \"prometheus-k8s-0\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:13:43.590325 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.590288    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/84e0c89a-ea89-4303-ab75-20cd7ddf0752-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:13:43.590432 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.590341    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/84e0c89a-ea89-4303-ab75-20cd7ddf0752-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:13:43.590432 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.590369    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/84e0c89a-ea89-4303-ab75-20cd7ddf0752-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:13:43.590432 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.590388    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/84e0c89a-ea89-4303-ab75-20cd7ddf0752-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:13:43.691592 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.691558    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84e0c89a-ea89-4303-ab75-20cd7ddf0752-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:13:43.691592 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.691598    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/84e0c89a-ea89-4303-ab75-20cd7ddf0752-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:13:43.691813 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.691615    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84e0c89a-ea89-4303-ab75-20cd7ddf0752-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:13:43.691813 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.691652    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/84e0c89a-ea89-4303-ab75-20cd7ddf0752-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:13:43.691813 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.691687    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/84e0c89a-ea89-4303-ab75-20cd7ddf0752-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:13:43.691813 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.691714    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/84e0c89a-ea89-4303-ab75-20cd7ddf0752-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:13:43.692004 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.691825    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/84e0c89a-ea89-4303-ab75-20cd7ddf0752-web-config\") pod \"prometheus-k8s-0\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:13:43.692004 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.691866    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/84e0c89a-ea89-4303-ab75-20cd7ddf0752-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:13:43.692004 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.691916    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/84e0c89a-ea89-4303-ab75-20cd7ddf0752-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:13:43.692004 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.691940    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/84e0c89a-ea89-4303-ab75-20cd7ddf0752-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:13:43.692004 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.691970    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/84e0c89a-ea89-4303-ab75-20cd7ddf0752-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:13:43.692250 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.692009    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/84e0c89a-ea89-4303-ab75-20cd7ddf0752-config\") pod \"prometheus-k8s-0\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:13:43.692250 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.692034    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/84e0c89a-ea89-4303-ab75-20cd7ddf0752-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:13:43.692250 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.692059    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/84e0c89a-ea89-4303-ab75-20cd7ddf0752-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:13:43.692250 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.692085    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/84e0c89a-ea89-4303-ab75-20cd7ddf0752-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:13:43.692250 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.692110    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/84e0c89a-ea89-4303-ab75-20cd7ddf0752-config-out\") pod \"prometheus-k8s-0\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:13:43.692250 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.692135    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84e0c89a-ea89-4303-ab75-20cd7ddf0752-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:13:43.692250 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.692163    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/84e0c89a-ea89-4303-ab75-20cd7ddf0752-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:13:43.692250 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.692202    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/84e0c89a-ea89-4303-ab75-20cd7ddf0752-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:13:43.692250 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.692230    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nw9s6\" (UniqueName: \"kubernetes.io/projected/84e0c89a-ea89-4303-ab75-20cd7ddf0752-kube-api-access-nw9s6\") pod \"prometheus-k8s-0\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:13:43.692741 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.692365    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84e0c89a-ea89-4303-ab75-20cd7ddf0752-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:13:43.692741 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.692487    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84e0c89a-ea89-4303-ab75-20cd7ddf0752-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:13:43.693650 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.693293    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/84e0c89a-ea89-4303-ab75-20cd7ddf0752-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:13:43.695036 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.695010    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84e0c89a-ea89-4303-ab75-20cd7ddf0752-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:13:43.696042 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.696013    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/84e0c89a-ea89-4303-ab75-20cd7ddf0752-config\") pod \"prometheus-k8s-0\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:13:43.696193 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.696168    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/84e0c89a-ea89-4303-ab75-20cd7ddf0752-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:13:43.696441 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.696419    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/84e0c89a-ea89-4303-ab75-20cd7ddf0752-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:13:43.696780 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.696758    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/84e0c89a-ea89-4303-ab75-20cd7ddf0752-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:13:43.697056 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.696760    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/84e0c89a-ea89-4303-ab75-20cd7ddf0752-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:13:43.697056 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.696973    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/84e0c89a-ea89-4303-ab75-20cd7ddf0752-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:13:43.697702 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.697670    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/84e0c89a-ea89-4303-ab75-20cd7ddf0752-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:13:43.697788 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.697759    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/84e0c89a-ea89-4303-ab75-20cd7ddf0752-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:13:43.698484 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.698213    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/84e0c89a-ea89-4303-ab75-20cd7ddf0752-web-config\") pod \"prometheus-k8s-0\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:13:43.698594 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.698571    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/84e0c89a-ea89-4303-ab75-20cd7ddf0752-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:13:43.698640 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.698595    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/84e0c89a-ea89-4303-ab75-20cd7ddf0752-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:13:43.698640 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.698571    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/84e0c89a-ea89-4303-ab75-20cd7ddf0752-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:13:43.698739 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.698711    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/84e0c89a-ea89-4303-ab75-20cd7ddf0752-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:13:43.698823 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.698810    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/84e0c89a-ea89-4303-ab75-20cd7ddf0752-config-out\") pod \"prometheus-k8s-0\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:13:43.699744 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.699724    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/84e0c89a-ea89-4303-ab75-20cd7ddf0752-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:13:43.704278 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.704257    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nw9s6\" (UniqueName: \"kubernetes.io/projected/84e0c89a-ea89-4303-ab75-20cd7ddf0752-kube-api-access-nw9s6\") pod \"prometheus-k8s-0\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:13:43.856242 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.856206    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:13:43.911826 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.911620    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-frbps"
Sep 09 16:13:43.957484 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.957255    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-c47648b58-qxzjc" event={"ID":"8be9e6b1-71f9-4055-914c-66b813d981ec","Type":"ContainerStarted","Data":"b53109f7e22455cdd59efade672d6fd70ee290bb823d5ac00b17858a90b7576a"}
Sep 09 16:13:43.977213 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:43.977148    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-c47648b58-qxzjc" podStartSLOduration=1.420103168 podStartE2EDuration="2.977122835s" totalImagesPullingTime="1.557019667s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 16:13:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 16:13:43.974798108 +0000 UTC m=+299.392648109" watchObservedRunningTime="2026-09-09 16:13:43.977122835 +0000 UTC m=+299.394972835"
Sep 09 16:13:44.147218 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:44.147190    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 09 16:13:44.149635 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:13:44.149609    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84e0c89a_ea89_4303_ab75_20cd7ddf0752.slice/crio-c00d5dafcf987b11868705fba886842a9ecbee048648652e4860f02615ee28c1 WatchSource:0}: Error finding container c00d5dafcf987b11868705fba886842a9ecbee048648652e4860f02615ee28c1: Status 404 returned error can't find the container with id c00d5dafcf987b11868705fba886842a9ecbee048648652e4860f02615ee28c1
Sep 09 16:13:44.962285 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:44.962246    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"84e0c89a-ea89-4303-ab75-20cd7ddf0752","Type":"ContainerStarted","Data":"c00d5dafcf987b11868705fba886842a9ecbee048648652e4860f02615ee28c1"}
Sep 09 16:13:45.023342 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:45.023216    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-xwlzs_16ecfb4d-7944-465e-a38e-d8c5bda6b1ac/console-operator/2.log"
Sep 09 16:13:45.023342 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:45.023262    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-xwlzs_16ecfb4d-7944-465e-a38e-d8c5bda6b1ac/console-operator/2.log"
Sep 09 16:13:45.035101 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:45.035070    2567 kubelet.go:1648] "Image garbage collection succeeded"
Sep 09 16:13:45.036201 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:45.036183    2567 scope.go:117] "RemoveContainer" containerID="40476ca069f118fa8524f1f7f48aafd5b8e5255ccb4228a14dbdf85ce792a657"
Sep 09 16:13:45.806892 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:45.806847    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7ccf8cd948-76htb" podUID="071d80b2-9f57-45c3-980d-678dc98590f1" containerName="registry" containerID="cri-o://1a5a9ec4f8fa7abafcd2fc27afcf0382ed5d1f3a1de2378c4eb1be7e94d886a1" gracePeriod=30
Sep 09 16:13:45.968618 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:45.968581    2567 generic.go:356] "Generic (PLEG): container finished" podID="84e0c89a-ea89-4303-ab75-20cd7ddf0752" containerID="4367e3d7420276b074cb71caae8ae96025e5e2134b741ba6b51ea812401cd3d5" exitCode=0
Sep 09 16:13:45.969086 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:45.968682    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"84e0c89a-ea89-4303-ab75-20cd7ddf0752","Type":"ContainerDied","Data":"4367e3d7420276b074cb71caae8ae96025e5e2134b741ba6b51ea812401cd3d5"}
Sep 09 16:13:45.971074 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:45.971041    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-xwlzs_16ecfb4d-7944-465e-a38e-d8c5bda6b1ac/console-operator/2.log"
Sep 09 16:13:45.971166 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:45.971132    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-xwlzs" event={"ID":"16ecfb4d-7944-465e-a38e-d8c5bda6b1ac","Type":"ContainerStarted","Data":"a9d38040a61100c681aa20619ca6b2beece48a5fab72ccc84f5340846f8da308"}
Sep 09 16:13:45.971440 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:45.971421    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-xwlzs"
Sep 09 16:13:45.972835 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:45.972807    2567 generic.go:356] "Generic (PLEG): container finished" podID="071d80b2-9f57-45c3-980d-678dc98590f1" containerID="1a5a9ec4f8fa7abafcd2fc27afcf0382ed5d1f3a1de2378c4eb1be7e94d886a1" exitCode=0
Sep 09 16:13:45.972950 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:45.972865    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7ccf8cd948-76htb" event={"ID":"071d80b2-9f57-45c3-980d-678dc98590f1","Type":"ContainerDied","Data":"1a5a9ec4f8fa7abafcd2fc27afcf0382ed5d1f3a1de2378c4eb1be7e94d886a1"}
Sep 09 16:13:45.983245 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:45.983217    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-85b454874b-xwlzs"
Sep 09 16:13:46.044848 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:46.044801    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-85b454874b-xwlzs" podStartSLOduration=51.05033657 podStartE2EDuration="55.044784983s" totalImagesPullingTime="3.994448413s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 16:12:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 16:13:46.044282673 +0000 UTC m=+301.462132682" watchObservedRunningTime="2026-09-09 16:13:46.044784983 +0000 UTC m=+301.462634982"
Sep 09 16:13:46.049355 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:46.049333    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7ccf8cd948-76htb"
Sep 09 16:13:46.116183 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:46.116145    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/071d80b2-9f57-45c3-980d-678dc98590f1-registry-certificates\") pod \"071d80b2-9f57-45c3-980d-678dc98590f1\" (UID: \"071d80b2-9f57-45c3-980d-678dc98590f1\") "
Sep 09 16:13:46.116347 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:46.116190    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/071d80b2-9f57-45c3-980d-678dc98590f1-trusted-ca\") pod \"071d80b2-9f57-45c3-980d-678dc98590f1\" (UID: \"071d80b2-9f57-45c3-980d-678dc98590f1\") "
Sep 09 16:13:46.116347 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:46.116235    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/071d80b2-9f57-45c3-980d-678dc98590f1-installation-pull-secrets\") pod \"071d80b2-9f57-45c3-980d-678dc98590f1\" (UID: \"071d80b2-9f57-45c3-980d-678dc98590f1\") "
Sep 09 16:13:46.116433 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:46.116339    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fdxvf\" (UniqueName: \"kubernetes.io/projected/071d80b2-9f57-45c3-980d-678dc98590f1-kube-api-access-fdxvf\") pod \"071d80b2-9f57-45c3-980d-678dc98590f1\" (UID: \"071d80b2-9f57-45c3-980d-678dc98590f1\") "
Sep 09 16:13:46.116433 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:46.116391    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/071d80b2-9f57-45c3-980d-678dc98590f1-bound-sa-token\") pod \"071d80b2-9f57-45c3-980d-678dc98590f1\" (UID: \"071d80b2-9f57-45c3-980d-678dc98590f1\") "
Sep 09 16:13:46.116433 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:46.116425    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/071d80b2-9f57-45c3-980d-678dc98590f1-registry-tls\") pod \"071d80b2-9f57-45c3-980d-678dc98590f1\" (UID: \"071d80b2-9f57-45c3-980d-678dc98590f1\") "
Sep 09 16:13:46.116607 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:46.116484    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/071d80b2-9f57-45c3-980d-678dc98590f1-image-registry-private-configuration\") pod \"071d80b2-9f57-45c3-980d-678dc98590f1\" (UID: \"071d80b2-9f57-45c3-980d-678dc98590f1\") "
Sep 09 16:13:46.116607 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:46.116531    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/071d80b2-9f57-45c3-980d-678dc98590f1-ca-trust-extracted\") pod \"071d80b2-9f57-45c3-980d-678dc98590f1\" (UID: \"071d80b2-9f57-45c3-980d-678dc98590f1\") "
Sep 09 16:13:46.116713 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:46.116667    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/071d80b2-9f57-45c3-980d-678dc98590f1-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "071d80b2-9f57-45c3-980d-678dc98590f1" (UID: "071d80b2-9f57-45c3-980d-678dc98590f1"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 16:13:46.117018 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:46.116993    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/071d80b2-9f57-45c3-980d-678dc98590f1-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "071d80b2-9f57-45c3-980d-678dc98590f1" (UID: "071d80b2-9f57-45c3-980d-678dc98590f1"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 16:13:46.118390 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:46.118355    2567 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/071d80b2-9f57-45c3-980d-678dc98590f1-registry-certificates\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:13:46.118740 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:46.118425    2567 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/071d80b2-9f57-45c3-980d-678dc98590f1-trusted-ca\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:13:46.119876 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:46.119834    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/071d80b2-9f57-45c3-980d-678dc98590f1-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "071d80b2-9f57-45c3-980d-678dc98590f1" (UID: "071d80b2-9f57-45c3-980d-678dc98590f1"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 16:13:46.120051 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:46.120026    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/071d80b2-9f57-45c3-980d-678dc98590f1-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "071d80b2-9f57-45c3-980d-678dc98590f1" (UID: "071d80b2-9f57-45c3-980d-678dc98590f1"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 16:13:46.120162 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:46.120128    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/071d80b2-9f57-45c3-980d-678dc98590f1-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "071d80b2-9f57-45c3-980d-678dc98590f1" (UID: "071d80b2-9f57-45c3-980d-678dc98590f1"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 16:13:46.120162 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:46.120140    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/071d80b2-9f57-45c3-980d-678dc98590f1-kube-api-access-fdxvf" (OuterVolumeSpecName: "kube-api-access-fdxvf") pod "071d80b2-9f57-45c3-980d-678dc98590f1" (UID: "071d80b2-9f57-45c3-980d-678dc98590f1"). InnerVolumeSpecName "kube-api-access-fdxvf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 16:13:46.120290 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:46.120219    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/071d80b2-9f57-45c3-980d-678dc98590f1-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "071d80b2-9f57-45c3-980d-678dc98590f1" (UID: "071d80b2-9f57-45c3-980d-678dc98590f1"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 16:13:46.125887 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:46.125866    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/071d80b2-9f57-45c3-980d-678dc98590f1-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "071d80b2-9f57-45c3-980d-678dc98590f1" (UID: "071d80b2-9f57-45c3-980d-678dc98590f1"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:13:46.219500 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:46.219444    2567 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/071d80b2-9f57-45c3-980d-678dc98590f1-ca-trust-extracted\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:13:46.219500 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:46.219493    2567 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/071d80b2-9f57-45c3-980d-678dc98590f1-installation-pull-secrets\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:13:46.219500 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:46.219503    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fdxvf\" (UniqueName: \"kubernetes.io/projected/071d80b2-9f57-45c3-980d-678dc98590f1-kube-api-access-fdxvf\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:13:46.219732 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:46.219511    2567 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/071d80b2-9f57-45c3-980d-678dc98590f1-bound-sa-token\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:13:46.219732 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:46.219521    2567 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/071d80b2-9f57-45c3-980d-678dc98590f1-registry-tls\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:13:46.219732 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:46.219529    2567 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/071d80b2-9f57-45c3-980d-678dc98590f1-image-registry-private-configuration\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:13:46.978409 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:46.978353    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7ccf8cd948-76htb" event={"ID":"071d80b2-9f57-45c3-980d-678dc98590f1","Type":"ContainerDied","Data":"3839fb044d23e69b793a7084204008ebbee59e4509cddd5f7707722ff2fe7f00"}
Sep 09 16:13:46.978915 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:46.978449    2567 scope.go:117] "RemoveContainer" containerID="1a5a9ec4f8fa7abafcd2fc27afcf0382ed5d1f3a1de2378c4eb1be7e94d886a1"
Sep 09 16:13:46.978915 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:46.978390    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7ccf8cd948-76htb"
Sep 09 16:13:47.002031 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:47.002003    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7ccf8cd948-76htb"]
Sep 09 16:13:47.007297 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:47.007270    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7ccf8cd948-76htb"]
Sep 09 16:13:47.036670 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:47.036635    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="071d80b2-9f57-45c3-980d-678dc98590f1" path="/var/lib/kubelet/pods/071d80b2-9f57-45c3-980d-678dc98590f1/volumes"
Sep 09 16:13:49.728910 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:49.728893    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 16:13:49.995315 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:49.995230    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"84e0c89a-ea89-4303-ab75-20cd7ddf0752","Type":"ContainerStarted","Data":"7f94864c04e1832d8f0c1811ec85741834cc999412cb6bcbfaca226f0bea5b75"}
Sep 09 16:13:49.995315 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:49.995293    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"84e0c89a-ea89-4303-ab75-20cd7ddf0752","Type":"ContainerStarted","Data":"2bffbadca619f0bee3d548669ab94af54128e45c8273aea9620d6f92024a1a20"}
Sep 09 16:13:55.017944 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:55.017905    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"84e0c89a-ea89-4303-ab75-20cd7ddf0752","Type":"ContainerStarted","Data":"8cc86096bc9d62618a2c3de6f3f23cc3f129848857e335f6dee03c929c1a91b0"}
Sep 09 16:13:55.018330 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:55.017975    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"84e0c89a-ea89-4303-ab75-20cd7ddf0752","Type":"ContainerStarted","Data":"29a120f9c840533949d3cbdeffc62e8b033c67aed2f1d0c8e2923effd6c83f4c"}
Sep 09 16:13:55.018330 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:55.018003    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"84e0c89a-ea89-4303-ab75-20cd7ddf0752","Type":"ContainerStarted","Data":"438a63a8ebb03df69558a54eb5bbd1c5d764fb092fc498fd386dec96bc1277c0"}
Sep 09 16:13:55.018330 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:55.018027    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"84e0c89a-ea89-4303-ab75-20cd7ddf0752","Type":"ContainerStarted","Data":"f801b58f4e4fbc7dcb86f49a37ee64e4fd4492072a17df4f3ca1f09b44ac5830"}
Sep 09 16:13:55.030149 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:55.030125    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t29nl"
Sep 09 16:13:55.049298 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:55.049236    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.737436477 podStartE2EDuration="12.049214315s" totalImagesPullingTime="9.293125081s" totalInitContainerRuntime="18.652757ms" isStatefulPod=false podCreationTimestamp="2026-09-09 16:13:43 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 16:13:55.043749657 +0000 UTC m=+310.461599677" watchObservedRunningTime="2026-09-09 16:13:55.049214315 +0000 UTC m=+310.467064313"
Sep 09 16:13:58.857064 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:13:58.857013    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:14:01.915285 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:14:01.915251    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-c47648b58-qxzjc"
Sep 09 16:14:01.915788 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:14:01.915331    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-c47648b58-qxzjc"
Sep 09 16:14:17.930646 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:14:17.930616    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7d7756ccd6-mfpk8_5ae67a1e-f25a-4218-a9ce-d4614dd65d43/router/0.log"
Sep 09 16:14:17.955947 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:14:17.955917    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tbdg5_65ebbfa4-b996-49d0-ae3e-2a1b8c3d7677/serve-healthcheck-canary/0.log"
Sep 09 16:14:21.920418 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:14:21.920388    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-c47648b58-qxzjc"
Sep 09 16:14:21.924231 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:14:21.924210    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-c47648b58-qxzjc"
Sep 09 16:14:43.856987 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:14:43.856888    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:14:43.873032 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:14:43.873007    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:14:44.216083 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:14:44.216016    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:14:57.926984 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:14:57.926944    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73b8ff58-aa37-4fab-9632-9b0f5d409902-metrics-certs\") pod \"network-metrics-daemon-t29nl\" (UID: \"73b8ff58-aa37-4fab-9632-9b0f5d409902\") " pod="openshift-multus/network-metrics-daemon-t29nl"
Sep 09 16:14:57.929327 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:14:57.929309    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73b8ff58-aa37-4fab-9632-9b0f5d409902-metrics-certs\") pod \"network-metrics-daemon-t29nl\" (UID: \"73b8ff58-aa37-4fab-9632-9b0f5d409902\") " pod="openshift-multus/network-metrics-daemon-t29nl"
Sep 09 16:14:58.033089 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:14:58.033054    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zj7bv\""
Sep 09 16:14:58.040844 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:14:58.040821    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t29nl"
Sep 09 16:14:58.314077 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:14:58.314004    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-t29nl"]
Sep 09 16:14:58.317092 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:14:58.317057    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73b8ff58_aa37_4fab_9632_9b0f5d409902.slice/crio-ce3dc457a743e230151abd07c461961a1cbdd66f9a42c199a9a06e00dae47529 WatchSource:0}: Error finding container ce3dc457a743e230151abd07c461961a1cbdd66f9a42c199a9a06e00dae47529: Status 404 returned error can't find the container with id ce3dc457a743e230151abd07c461961a1cbdd66f9a42c199a9a06e00dae47529
Sep 09 16:14:59.257160 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:14:59.257105    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-t29nl" event={"ID":"73b8ff58-aa37-4fab-9632-9b0f5d409902","Type":"ContainerStarted","Data":"ce3dc457a743e230151abd07c461961a1cbdd66f9a42c199a9a06e00dae47529"}
Sep 09 16:15:00.263101 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:00.263068    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-t29nl" event={"ID":"73b8ff58-aa37-4fab-9632-9b0f5d409902","Type":"ContainerStarted","Data":"da316e75e5c8c319e83edb4c0614c68ef7ec95ec44cb161afc0365b9e5e94571"}
Sep 09 16:15:00.263529 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:00.263122    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-t29nl" event={"ID":"73b8ff58-aa37-4fab-9632-9b0f5d409902","Type":"ContainerStarted","Data":"0431921d0554baf348d07029100776cfaac555bd18d0ce3c4827fbae601f1e79"}
Sep 09 16:15:00.282558 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:00.282502    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-t29nl" podStartSLOduration=374.150558737 podStartE2EDuration="6m15.282478645s" totalImagesPullingTime="1.131919908s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 16:08:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 16:15:00.281704736 +0000 UTC m=+375.699554754" watchObservedRunningTime="2026-09-09 16:15:00.282478645 +0000 UTC m=+375.700328662"
Sep 09 16:15:01.994821 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:01.994788    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 09 16:15:01.995533 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:01.995284    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="84e0c89a-ea89-4303-ab75-20cd7ddf0752" containerName="prometheus" containerID="cri-o://2bffbadca619f0bee3d548669ab94af54128e45c8273aea9620d6f92024a1a20" gracePeriod=600
Sep 09 16:15:01.995650 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:01.995355    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="84e0c89a-ea89-4303-ab75-20cd7ddf0752" containerName="kube-rbac-proxy-web" containerID="cri-o://438a63a8ebb03df69558a54eb5bbd1c5d764fb092fc498fd386dec96bc1277c0" gracePeriod=600
Sep 09 16:15:01.995756 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:01.995373    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="84e0c89a-ea89-4303-ab75-20cd7ddf0752" containerName="kube-rbac-proxy" containerID="cri-o://29a120f9c840533949d3cbdeffc62e8b033c67aed2f1d0c8e2923effd6c83f4c" gracePeriod=600
Sep 09 16:15:01.995840 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:01.995398    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="84e0c89a-ea89-4303-ab75-20cd7ddf0752" containerName="thanos-sidecar" containerID="cri-o://f801b58f4e4fbc7dcb86f49a37ee64e4fd4492072a17df4f3ca1f09b44ac5830" gracePeriod=600
Sep 09 16:15:01.995903 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:01.995411    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="84e0c89a-ea89-4303-ab75-20cd7ddf0752" containerName="config-reloader" containerID="cri-o://7f94864c04e1832d8f0c1811ec85741834cc999412cb6bcbfaca226f0bea5b75" gracePeriod=600
Sep 09 16:15:01.995976 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:01.995830    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="84e0c89a-ea89-4303-ab75-20cd7ddf0752" containerName="kube-rbac-proxy-thanos" containerID="cri-o://8cc86096bc9d62618a2c3de6f3f23cc3f129848857e335f6dee03c929c1a91b0" gracePeriod=600
Sep 09 16:15:02.244897 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.244836    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:15:02.256807 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.256774    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84e0c89a-ea89-4303-ab75-20cd7ddf0752-configmap-serving-certs-ca-bundle\") pod \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") "
Sep 09 16:15:02.256947 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.256820    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/84e0c89a-ea89-4303-ab75-20cd7ddf0752-secret-kube-rbac-proxy\") pod \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") "
Sep 09 16:15:02.256947 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.256849    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/84e0c89a-ea89-4303-ab75-20cd7ddf0752-secret-metrics-client-certs\") pod \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") "
Sep 09 16:15:02.256947 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.256885    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/84e0c89a-ea89-4303-ab75-20cd7ddf0752-config\") pod \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") "
Sep 09 16:15:02.256947 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.256909    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/84e0c89a-ea89-4303-ab75-20cd7ddf0752-web-config\") pod \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") "
Sep 09 16:15:02.257168 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.256955    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/84e0c89a-ea89-4303-ab75-20cd7ddf0752-prometheus-k8s-rulefiles-1\") pod \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") "
Sep 09 16:15:02.257168 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.256979    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84e0c89a-ea89-4303-ab75-20cd7ddf0752-prometheus-trusted-ca-bundle\") pod \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") "
Sep 09 16:15:02.257168 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.257004    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/84e0c89a-ea89-4303-ab75-20cd7ddf0752-secret-grpc-tls\") pod \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") "
Sep 09 16:15:02.257168 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.257031    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nw9s6\" (UniqueName: \"kubernetes.io/projected/84e0c89a-ea89-4303-ab75-20cd7ddf0752-kube-api-access-nw9s6\") pod \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") "
Sep 09 16:15:02.257168 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.257054    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/84e0c89a-ea89-4303-ab75-20cd7ddf0752-thanos-prometheus-http-client-file\") pod \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") "
Sep 09 16:15:02.257168 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.257095    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/84e0c89a-ea89-4303-ab75-20cd7ddf0752-configmap-metrics-client-ca\") pod \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") "
Sep 09 16:15:02.257168 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.257122    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/84e0c89a-ea89-4303-ab75-20cd7ddf0752-secret-prometheus-k8s-tls\") pod \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") "
Sep 09 16:15:02.257168 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.257149    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/84e0c89a-ea89-4303-ab75-20cd7ddf0752-prometheus-k8s-db\") pod \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") "
Sep 09 16:15:02.257570 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.257180    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84e0c89a-ea89-4303-ab75-20cd7ddf0752-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "84e0c89a-ea89-4303-ab75-20cd7ddf0752" (UID: "84e0c89a-ea89-4303-ab75-20cd7ddf0752"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 16:15:02.257570 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.257192    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84e0c89a-ea89-4303-ab75-20cd7ddf0752-configmap-kubelet-serving-ca-bundle\") pod \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") "
Sep 09 16:15:02.257570 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.257265    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/84e0c89a-ea89-4303-ab75-20cd7ddf0752-config-out\") pod \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") "
Sep 09 16:15:02.257570 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.257295    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/84e0c89a-ea89-4303-ab75-20cd7ddf0752-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") "
Sep 09 16:15:02.257570 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.257326    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/84e0c89a-ea89-4303-ab75-20cd7ddf0752-tls-assets\") pod \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") "
Sep 09 16:15:02.257570 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.257353    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/84e0c89a-ea89-4303-ab75-20cd7ddf0752-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") "
Sep 09 16:15:02.257570 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.257380    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/84e0c89a-ea89-4303-ab75-20cd7ddf0752-prometheus-k8s-rulefiles-2\") pod \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") "
Sep 09 16:15:02.257570 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.257418    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/84e0c89a-ea89-4303-ab75-20cd7ddf0752-prometheus-k8s-rulefiles-0\") pod \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\" (UID: \"84e0c89a-ea89-4303-ab75-20cd7ddf0752\") "
Sep 09 16:15:02.257959 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.257579    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84e0c89a-ea89-4303-ab75-20cd7ddf0752-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "84e0c89a-ea89-4303-ab75-20cd7ddf0752" (UID: "84e0c89a-ea89-4303-ab75-20cd7ddf0752"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 16:15:02.257959 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.257871    2567 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84e0c89a-ea89-4303-ab75-20cd7ddf0752-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:15:02.257959 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.257893    2567 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84e0c89a-ea89-4303-ab75-20cd7ddf0752-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:15:02.260385 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.258679    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84e0c89a-ea89-4303-ab75-20cd7ddf0752-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "84e0c89a-ea89-4303-ab75-20cd7ddf0752" (UID: "84e0c89a-ea89-4303-ab75-20cd7ddf0752"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 16:15:02.260385 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.258996    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84e0c89a-ea89-4303-ab75-20cd7ddf0752-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "84e0c89a-ea89-4303-ab75-20cd7ddf0752" (UID: "84e0c89a-ea89-4303-ab75-20cd7ddf0752"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 16:15:02.260385 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.259195    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84e0c89a-ea89-4303-ab75-20cd7ddf0752-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "84e0c89a-ea89-4303-ab75-20cd7ddf0752" (UID: "84e0c89a-ea89-4303-ab75-20cd7ddf0752"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 16:15:02.260385 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.259386    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84e0c89a-ea89-4303-ab75-20cd7ddf0752-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "84e0c89a-ea89-4303-ab75-20cd7ddf0752" (UID: "84e0c89a-ea89-4303-ab75-20cd7ddf0752"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 16:15:02.260385 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.260350    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84e0c89a-ea89-4303-ab75-20cd7ddf0752-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "84e0c89a-ea89-4303-ab75-20cd7ddf0752" (UID: "84e0c89a-ea89-4303-ab75-20cd7ddf0752"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:15:02.260786 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.260396    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84e0c89a-ea89-4303-ab75-20cd7ddf0752-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "84e0c89a-ea89-4303-ab75-20cd7ddf0752" (UID: "84e0c89a-ea89-4303-ab75-20cd7ddf0752"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 16:15:02.260838 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.260823    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84e0c89a-ea89-4303-ab75-20cd7ddf0752-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "84e0c89a-ea89-4303-ab75-20cd7ddf0752" (UID: "84e0c89a-ea89-4303-ab75-20cd7ddf0752"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 16:15:02.261940 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.261915    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84e0c89a-ea89-4303-ab75-20cd7ddf0752-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "84e0c89a-ea89-4303-ab75-20cd7ddf0752" (UID: "84e0c89a-ea89-4303-ab75-20cd7ddf0752"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 16:15:02.262064 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.262034    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84e0c89a-ea89-4303-ab75-20cd7ddf0752-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "84e0c89a-ea89-4303-ab75-20cd7ddf0752" (UID: "84e0c89a-ea89-4303-ab75-20cd7ddf0752"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 16:15:02.262300 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.262276    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84e0c89a-ea89-4303-ab75-20cd7ddf0752-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "84e0c89a-ea89-4303-ab75-20cd7ddf0752" (UID: "84e0c89a-ea89-4303-ab75-20cd7ddf0752"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 16:15:02.262416 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.262389    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84e0c89a-ea89-4303-ab75-20cd7ddf0752-config" (OuterVolumeSpecName: "config") pod "84e0c89a-ea89-4303-ab75-20cd7ddf0752" (UID: "84e0c89a-ea89-4303-ab75-20cd7ddf0752"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 16:15:02.262416 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.262407    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84e0c89a-ea89-4303-ab75-20cd7ddf0752-config-out" (OuterVolumeSpecName: "config-out") pod "84e0c89a-ea89-4303-ab75-20cd7ddf0752" (UID: "84e0c89a-ea89-4303-ab75-20cd7ddf0752"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:15:02.262629 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.262565    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84e0c89a-ea89-4303-ab75-20cd7ddf0752-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "84e0c89a-ea89-4303-ab75-20cd7ddf0752" (UID: "84e0c89a-ea89-4303-ab75-20cd7ddf0752"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 16:15:02.263111 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.263083    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84e0c89a-ea89-4303-ab75-20cd7ddf0752-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "84e0c89a-ea89-4303-ab75-20cd7ddf0752" (UID: "84e0c89a-ea89-4303-ab75-20cd7ddf0752"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 16:15:02.263622 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.263582    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84e0c89a-ea89-4303-ab75-20cd7ddf0752-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "84e0c89a-ea89-4303-ab75-20cd7ddf0752" (UID: "84e0c89a-ea89-4303-ab75-20cd7ddf0752"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 16:15:02.263865 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.263800    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84e0c89a-ea89-4303-ab75-20cd7ddf0752-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "84e0c89a-ea89-4303-ab75-20cd7ddf0752" (UID: "84e0c89a-ea89-4303-ab75-20cd7ddf0752"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 16:15:02.263865 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.263827    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84e0c89a-ea89-4303-ab75-20cd7ddf0752-kube-api-access-nw9s6" (OuterVolumeSpecName: "kube-api-access-nw9s6") pod "84e0c89a-ea89-4303-ab75-20cd7ddf0752" (UID: "84e0c89a-ea89-4303-ab75-20cd7ddf0752"). InnerVolumeSpecName "kube-api-access-nw9s6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 16:15:02.278133 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.278098    2567 generic.go:356] "Generic (PLEG): container finished" podID="84e0c89a-ea89-4303-ab75-20cd7ddf0752" containerID="8cc86096bc9d62618a2c3de6f3f23cc3f129848857e335f6dee03c929c1a91b0" exitCode=0
Sep 09 16:15:02.278133 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.278129    2567 generic.go:356] "Generic (PLEG): container finished" podID="84e0c89a-ea89-4303-ab75-20cd7ddf0752" containerID="29a120f9c840533949d3cbdeffc62e8b033c67aed2f1d0c8e2923effd6c83f4c" exitCode=0
Sep 09 16:15:02.278133 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.278137    2567 generic.go:356] "Generic (PLEG): container finished" podID="84e0c89a-ea89-4303-ab75-20cd7ddf0752" containerID="438a63a8ebb03df69558a54eb5bbd1c5d764fb092fc498fd386dec96bc1277c0" exitCode=0
Sep 09 16:15:02.278362 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.278148    2567 generic.go:356] "Generic (PLEG): container finished" podID="84e0c89a-ea89-4303-ab75-20cd7ddf0752" containerID="f801b58f4e4fbc7dcb86f49a37ee64e4fd4492072a17df4f3ca1f09b44ac5830" exitCode=0
Sep 09 16:15:02.278362 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.278156    2567 generic.go:356] "Generic (PLEG): container finished" podID="84e0c89a-ea89-4303-ab75-20cd7ddf0752" containerID="7f94864c04e1832d8f0c1811ec85741834cc999412cb6bcbfaca226f0bea5b75" exitCode=0
Sep 09 16:15:02.278362 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.278164    2567 generic.go:356] "Generic (PLEG): container finished" podID="84e0c89a-ea89-4303-ab75-20cd7ddf0752" containerID="2bffbadca619f0bee3d548669ab94af54128e45c8273aea9620d6f92024a1a20" exitCode=0
Sep 09 16:15:02.278362 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.278210    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"84e0c89a-ea89-4303-ab75-20cd7ddf0752","Type":"ContainerDied","Data":"8cc86096bc9d62618a2c3de6f3f23cc3f129848857e335f6dee03c929c1a91b0"}
Sep 09 16:15:02.278362 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.278311    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"84e0c89a-ea89-4303-ab75-20cd7ddf0752","Type":"ContainerDied","Data":"29a120f9c840533949d3cbdeffc62e8b033c67aed2f1d0c8e2923effd6c83f4c"}
Sep 09 16:15:02.278362 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.278353    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"84e0c89a-ea89-4303-ab75-20cd7ddf0752","Type":"ContainerDied","Data":"438a63a8ebb03df69558a54eb5bbd1c5d764fb092fc498fd386dec96bc1277c0"}
Sep 09 16:15:02.278678 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.278421    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"84e0c89a-ea89-4303-ab75-20cd7ddf0752","Type":"ContainerDied","Data":"f801b58f4e4fbc7dcb86f49a37ee64e4fd4492072a17df4f3ca1f09b44ac5830"}
Sep 09 16:15:02.278678 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.278449    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"84e0c89a-ea89-4303-ab75-20cd7ddf0752","Type":"ContainerDied","Data":"7f94864c04e1832d8f0c1811ec85741834cc999412cb6bcbfaca226f0bea5b75"}
Sep 09 16:15:02.278678 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.278475    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:15:02.278678 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.278507    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"84e0c89a-ea89-4303-ab75-20cd7ddf0752","Type":"ContainerDied","Data":"2bffbadca619f0bee3d548669ab94af54128e45c8273aea9620d6f92024a1a20"}
Sep 09 16:15:02.278678 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.278534    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"84e0c89a-ea89-4303-ab75-20cd7ddf0752","Type":"ContainerDied","Data":"c00d5dafcf987b11868705fba886842a9ecbee048648652e4860f02615ee28c1"}
Sep 09 16:15:02.278678 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.278578    2567 scope.go:117] "RemoveContainer" containerID="8cc86096bc9d62618a2c3de6f3f23cc3f129848857e335f6dee03c929c1a91b0"
Sep 09 16:15:02.281409 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.281383    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84e0c89a-ea89-4303-ab75-20cd7ddf0752-web-config" (OuterVolumeSpecName: "web-config") pod "84e0c89a-ea89-4303-ab75-20cd7ddf0752" (UID: "84e0c89a-ea89-4303-ab75-20cd7ddf0752"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 16:15:02.290171 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.290152    2567 scope.go:117] "RemoveContainer" containerID="29a120f9c840533949d3cbdeffc62e8b033c67aed2f1d0c8e2923effd6c83f4c"
Sep 09 16:15:02.300060 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.300040    2567 scope.go:117] "RemoveContainer" containerID="438a63a8ebb03df69558a54eb5bbd1c5d764fb092fc498fd386dec96bc1277c0"
Sep 09 16:15:02.309195 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.309170    2567 scope.go:117] "RemoveContainer" containerID="f801b58f4e4fbc7dcb86f49a37ee64e4fd4492072a17df4f3ca1f09b44ac5830"
Sep 09 16:15:02.318303 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.318282    2567 scope.go:117] "RemoveContainer" containerID="7f94864c04e1832d8f0c1811ec85741834cc999412cb6bcbfaca226f0bea5b75"
Sep 09 16:15:02.328002 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.327977    2567 scope.go:117] "RemoveContainer" containerID="2bffbadca619f0bee3d548669ab94af54128e45c8273aea9620d6f92024a1a20"
Sep 09 16:15:02.337089 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.337058    2567 scope.go:117] "RemoveContainer" containerID="4367e3d7420276b074cb71caae8ae96025e5e2134b741ba6b51ea812401cd3d5"
Sep 09 16:15:02.344720 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.344702    2567 scope.go:117] "RemoveContainer" containerID="8cc86096bc9d62618a2c3de6f3f23cc3f129848857e335f6dee03c929c1a91b0"
Sep 09 16:15:02.345034 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:15:02.345001    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8cc86096bc9d62618a2c3de6f3f23cc3f129848857e335f6dee03c929c1a91b0\": container with ID starting with 8cc86096bc9d62618a2c3de6f3f23cc3f129848857e335f6dee03c929c1a91b0 not found: ID does not exist" containerID="8cc86096bc9d62618a2c3de6f3f23cc3f129848857e335f6dee03c929c1a91b0"
Sep 09 16:15:02.345117 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.345042    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8cc86096bc9d62618a2c3de6f3f23cc3f129848857e335f6dee03c929c1a91b0"} err="failed to get container status \"8cc86096bc9d62618a2c3de6f3f23cc3f129848857e335f6dee03c929c1a91b0\": rpc error: code = NotFound desc = could not find container \"8cc86096bc9d62618a2c3de6f3f23cc3f129848857e335f6dee03c929c1a91b0\": container with ID starting with 8cc86096bc9d62618a2c3de6f3f23cc3f129848857e335f6dee03c929c1a91b0 not found: ID does not exist"
Sep 09 16:15:02.345117 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.345080    2567 scope.go:117] "RemoveContainer" containerID="29a120f9c840533949d3cbdeffc62e8b033c67aed2f1d0c8e2923effd6c83f4c"
Sep 09 16:15:02.345384 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:15:02.345364    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29a120f9c840533949d3cbdeffc62e8b033c67aed2f1d0c8e2923effd6c83f4c\": container with ID starting with 29a120f9c840533949d3cbdeffc62e8b033c67aed2f1d0c8e2923effd6c83f4c not found: ID does not exist" containerID="29a120f9c840533949d3cbdeffc62e8b033c67aed2f1d0c8e2923effd6c83f4c"
Sep 09 16:15:02.345448 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.345391    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29a120f9c840533949d3cbdeffc62e8b033c67aed2f1d0c8e2923effd6c83f4c"} err="failed to get container status \"29a120f9c840533949d3cbdeffc62e8b033c67aed2f1d0c8e2923effd6c83f4c\": rpc error: code = NotFound desc = could not find container \"29a120f9c840533949d3cbdeffc62e8b033c67aed2f1d0c8e2923effd6c83f4c\": container with ID starting with 29a120f9c840533949d3cbdeffc62e8b033c67aed2f1d0c8e2923effd6c83f4c not found: ID does not exist"
Sep 09 16:15:02.345448 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.345410    2567 scope.go:117] "RemoveContainer" containerID="438a63a8ebb03df69558a54eb5bbd1c5d764fb092fc498fd386dec96bc1277c0"
Sep 09 16:15:02.345770 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:15:02.345747    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"438a63a8ebb03df69558a54eb5bbd1c5d764fb092fc498fd386dec96bc1277c0\": container with ID starting with 438a63a8ebb03df69558a54eb5bbd1c5d764fb092fc498fd386dec96bc1277c0 not found: ID does not exist" containerID="438a63a8ebb03df69558a54eb5bbd1c5d764fb092fc498fd386dec96bc1277c0"
Sep 09 16:15:02.345862 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.345777    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"438a63a8ebb03df69558a54eb5bbd1c5d764fb092fc498fd386dec96bc1277c0"} err="failed to get container status \"438a63a8ebb03df69558a54eb5bbd1c5d764fb092fc498fd386dec96bc1277c0\": rpc error: code = NotFound desc = could not find container \"438a63a8ebb03df69558a54eb5bbd1c5d764fb092fc498fd386dec96bc1277c0\": container with ID starting with 438a63a8ebb03df69558a54eb5bbd1c5d764fb092fc498fd386dec96bc1277c0 not found: ID does not exist"
Sep 09 16:15:02.345862 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.345795    2567 scope.go:117] "RemoveContainer" containerID="f801b58f4e4fbc7dcb86f49a37ee64e4fd4492072a17df4f3ca1f09b44ac5830"
Sep 09 16:15:02.346048 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:15:02.346032    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f801b58f4e4fbc7dcb86f49a37ee64e4fd4492072a17df4f3ca1f09b44ac5830\": container with ID starting with f801b58f4e4fbc7dcb86f49a37ee64e4fd4492072a17df4f3ca1f09b44ac5830 not found: ID does not exist" containerID="f801b58f4e4fbc7dcb86f49a37ee64e4fd4492072a17df4f3ca1f09b44ac5830"
Sep 09 16:15:02.346111 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.346050    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f801b58f4e4fbc7dcb86f49a37ee64e4fd4492072a17df4f3ca1f09b44ac5830"} err="failed to get container status \"f801b58f4e4fbc7dcb86f49a37ee64e4fd4492072a17df4f3ca1f09b44ac5830\": rpc error: code = NotFound desc = could not find container \"f801b58f4e4fbc7dcb86f49a37ee64e4fd4492072a17df4f3ca1f09b44ac5830\": container with ID starting with f801b58f4e4fbc7dcb86f49a37ee64e4fd4492072a17df4f3ca1f09b44ac5830 not found: ID does not exist"
Sep 09 16:15:02.346111 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.346062    2567 scope.go:117] "RemoveContainer" containerID="7f94864c04e1832d8f0c1811ec85741834cc999412cb6bcbfaca226f0bea5b75"
Sep 09 16:15:02.346290 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:15:02.346276    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f94864c04e1832d8f0c1811ec85741834cc999412cb6bcbfaca226f0bea5b75\": container with ID starting with 7f94864c04e1832d8f0c1811ec85741834cc999412cb6bcbfaca226f0bea5b75 not found: ID does not exist" containerID="7f94864c04e1832d8f0c1811ec85741834cc999412cb6bcbfaca226f0bea5b75"
Sep 09 16:15:02.346330 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.346293    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f94864c04e1832d8f0c1811ec85741834cc999412cb6bcbfaca226f0bea5b75"} err="failed to get container status \"7f94864c04e1832d8f0c1811ec85741834cc999412cb6bcbfaca226f0bea5b75\": rpc error: code = NotFound desc = could not find container \"7f94864c04e1832d8f0c1811ec85741834cc999412cb6bcbfaca226f0bea5b75\": container with ID starting with 7f94864c04e1832d8f0c1811ec85741834cc999412cb6bcbfaca226f0bea5b75 not found: ID does not exist"
Sep 09 16:15:02.346330 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.346305    2567 scope.go:117] "RemoveContainer" containerID="2bffbadca619f0bee3d548669ab94af54128e45c8273aea9620d6f92024a1a20"
Sep 09 16:15:02.346620 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:15:02.346603    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2bffbadca619f0bee3d548669ab94af54128e45c8273aea9620d6f92024a1a20\": container with ID starting with 2bffbadca619f0bee3d548669ab94af54128e45c8273aea9620d6f92024a1a20 not found: ID does not exist" containerID="2bffbadca619f0bee3d548669ab94af54128e45c8273aea9620d6f92024a1a20"
Sep 09 16:15:02.346671 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.346624    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2bffbadca619f0bee3d548669ab94af54128e45c8273aea9620d6f92024a1a20"} err="failed to get container status \"2bffbadca619f0bee3d548669ab94af54128e45c8273aea9620d6f92024a1a20\": rpc error: code = NotFound desc = could not find container \"2bffbadca619f0bee3d548669ab94af54128e45c8273aea9620d6f92024a1a20\": container with ID starting with 2bffbadca619f0bee3d548669ab94af54128e45c8273aea9620d6f92024a1a20 not found: ID does not exist"
Sep 09 16:15:02.346671 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.346636    2567 scope.go:117] "RemoveContainer" containerID="4367e3d7420276b074cb71caae8ae96025e5e2134b741ba6b51ea812401cd3d5"
Sep 09 16:15:02.346889 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:15:02.346872    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4367e3d7420276b074cb71caae8ae96025e5e2134b741ba6b51ea812401cd3d5\": container with ID starting with 4367e3d7420276b074cb71caae8ae96025e5e2134b741ba6b51ea812401cd3d5 not found: ID does not exist" containerID="4367e3d7420276b074cb71caae8ae96025e5e2134b741ba6b51ea812401cd3d5"
Sep 09 16:15:02.346953 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.346889    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4367e3d7420276b074cb71caae8ae96025e5e2134b741ba6b51ea812401cd3d5"} err="failed to get container status \"4367e3d7420276b074cb71caae8ae96025e5e2134b741ba6b51ea812401cd3d5\": rpc error: code = NotFound desc = could not find container \"4367e3d7420276b074cb71caae8ae96025e5e2134b741ba6b51ea812401cd3d5\": container with ID starting with 4367e3d7420276b074cb71caae8ae96025e5e2134b741ba6b51ea812401cd3d5 not found: ID does not exist"
Sep 09 16:15:02.346953 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.346902    2567 scope.go:117] "RemoveContainer" containerID="8cc86096bc9d62618a2c3de6f3f23cc3f129848857e335f6dee03c929c1a91b0"
Sep 09 16:15:02.347186 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.347168    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8cc86096bc9d62618a2c3de6f3f23cc3f129848857e335f6dee03c929c1a91b0"} err="failed to get container status \"8cc86096bc9d62618a2c3de6f3f23cc3f129848857e335f6dee03c929c1a91b0\": rpc error: code = NotFound desc = could not find container \"8cc86096bc9d62618a2c3de6f3f23cc3f129848857e335f6dee03c929c1a91b0\": container with ID starting with 8cc86096bc9d62618a2c3de6f3f23cc3f129848857e335f6dee03c929c1a91b0 not found: ID does not exist"
Sep 09 16:15:02.347241 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.347187    2567 scope.go:117] "RemoveContainer" containerID="29a120f9c840533949d3cbdeffc62e8b033c67aed2f1d0c8e2923effd6c83f4c"
Sep 09 16:15:02.347407 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.347390    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29a120f9c840533949d3cbdeffc62e8b033c67aed2f1d0c8e2923effd6c83f4c"} err="failed to get container status \"29a120f9c840533949d3cbdeffc62e8b033c67aed2f1d0c8e2923effd6c83f4c\": rpc error: code = NotFound desc = could not find container \"29a120f9c840533949d3cbdeffc62e8b033c67aed2f1d0c8e2923effd6c83f4c\": container with ID starting with 29a120f9c840533949d3cbdeffc62e8b033c67aed2f1d0c8e2923effd6c83f4c not found: ID does not exist"
Sep 09 16:15:02.347477 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.347407    2567 scope.go:117] "RemoveContainer" containerID="438a63a8ebb03df69558a54eb5bbd1c5d764fb092fc498fd386dec96bc1277c0"
Sep 09 16:15:02.347719 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.347701    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"438a63a8ebb03df69558a54eb5bbd1c5d764fb092fc498fd386dec96bc1277c0"} err="failed to get container status \"438a63a8ebb03df69558a54eb5bbd1c5d764fb092fc498fd386dec96bc1277c0\": rpc error: code = NotFound desc = could not find container \"438a63a8ebb03df69558a54eb5bbd1c5d764fb092fc498fd386dec96bc1277c0\": container with ID starting with 438a63a8ebb03df69558a54eb5bbd1c5d764fb092fc498fd386dec96bc1277c0 not found: ID does not exist"
Sep 09 16:15:02.347792 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.347719    2567 scope.go:117] "RemoveContainer" containerID="f801b58f4e4fbc7dcb86f49a37ee64e4fd4492072a17df4f3ca1f09b44ac5830"
Sep 09 16:15:02.348002 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.347980    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f801b58f4e4fbc7dcb86f49a37ee64e4fd4492072a17df4f3ca1f09b44ac5830"} err="failed to get container status \"f801b58f4e4fbc7dcb86f49a37ee64e4fd4492072a17df4f3ca1f09b44ac5830\": rpc error: code = NotFound desc = could not find container \"f801b58f4e4fbc7dcb86f49a37ee64e4fd4492072a17df4f3ca1f09b44ac5830\": container with ID starting with f801b58f4e4fbc7dcb86f49a37ee64e4fd4492072a17df4f3ca1f09b44ac5830 not found: ID does not exist"
Sep 09 16:15:02.348002 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.348002    2567 scope.go:117] "RemoveContainer" containerID="7f94864c04e1832d8f0c1811ec85741834cc999412cb6bcbfaca226f0bea5b75"
Sep 09 16:15:02.348275 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.348259    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f94864c04e1832d8f0c1811ec85741834cc999412cb6bcbfaca226f0bea5b75"} err="failed to get container status \"7f94864c04e1832d8f0c1811ec85741834cc999412cb6bcbfaca226f0bea5b75\": rpc error: code = NotFound desc = could not find container \"7f94864c04e1832d8f0c1811ec85741834cc999412cb6bcbfaca226f0bea5b75\": container with ID starting with 7f94864c04e1832d8f0c1811ec85741834cc999412cb6bcbfaca226f0bea5b75 not found: ID does not exist"
Sep 09 16:15:02.348322 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.348276    2567 scope.go:117] "RemoveContainer" containerID="2bffbadca619f0bee3d548669ab94af54128e45c8273aea9620d6f92024a1a20"
Sep 09 16:15:02.348542 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.348521    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2bffbadca619f0bee3d548669ab94af54128e45c8273aea9620d6f92024a1a20"} err="failed to get container status \"2bffbadca619f0bee3d548669ab94af54128e45c8273aea9620d6f92024a1a20\": rpc error: code = NotFound desc = could not find container \"2bffbadca619f0bee3d548669ab94af54128e45c8273aea9620d6f92024a1a20\": container with ID starting with 2bffbadca619f0bee3d548669ab94af54128e45c8273aea9620d6f92024a1a20 not found: ID does not exist"
Sep 09 16:15:02.348590 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.348545    2567 scope.go:117] "RemoveContainer" containerID="4367e3d7420276b074cb71caae8ae96025e5e2134b741ba6b51ea812401cd3d5"
Sep 09 16:15:02.348794 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.348775    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4367e3d7420276b074cb71caae8ae96025e5e2134b741ba6b51ea812401cd3d5"} err="failed to get container status \"4367e3d7420276b074cb71caae8ae96025e5e2134b741ba6b51ea812401cd3d5\": rpc error: code = NotFound desc = could not find container \"4367e3d7420276b074cb71caae8ae96025e5e2134b741ba6b51ea812401cd3d5\": container with ID starting with 4367e3d7420276b074cb71caae8ae96025e5e2134b741ba6b51ea812401cd3d5 not found: ID does not exist"
Sep 09 16:15:02.348858 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.348798    2567 scope.go:117] "RemoveContainer" containerID="8cc86096bc9d62618a2c3de6f3f23cc3f129848857e335f6dee03c929c1a91b0"
Sep 09 16:15:02.349049 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.349032    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8cc86096bc9d62618a2c3de6f3f23cc3f129848857e335f6dee03c929c1a91b0"} err="failed to get container status \"8cc86096bc9d62618a2c3de6f3f23cc3f129848857e335f6dee03c929c1a91b0\": rpc error: code = NotFound desc = could not find container \"8cc86096bc9d62618a2c3de6f3f23cc3f129848857e335f6dee03c929c1a91b0\": container with ID starting with 8cc86096bc9d62618a2c3de6f3f23cc3f129848857e335f6dee03c929c1a91b0 not found: ID does not exist"
Sep 09 16:15:02.349100 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.349049    2567 scope.go:117] "RemoveContainer" containerID="29a120f9c840533949d3cbdeffc62e8b033c67aed2f1d0c8e2923effd6c83f4c"
Sep 09 16:15:02.349269 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.349254    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29a120f9c840533949d3cbdeffc62e8b033c67aed2f1d0c8e2923effd6c83f4c"} err="failed to get container status \"29a120f9c840533949d3cbdeffc62e8b033c67aed2f1d0c8e2923effd6c83f4c\": rpc error: code = NotFound desc = could not find container \"29a120f9c840533949d3cbdeffc62e8b033c67aed2f1d0c8e2923effd6c83f4c\": container with ID starting with 29a120f9c840533949d3cbdeffc62e8b033c67aed2f1d0c8e2923effd6c83f4c not found: ID does not exist"
Sep 09 16:15:02.349324 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.349270    2567 scope.go:117] "RemoveContainer" containerID="438a63a8ebb03df69558a54eb5bbd1c5d764fb092fc498fd386dec96bc1277c0"
Sep 09 16:15:02.349569 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.349551    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"438a63a8ebb03df69558a54eb5bbd1c5d764fb092fc498fd386dec96bc1277c0"} err="failed to get container status \"438a63a8ebb03df69558a54eb5bbd1c5d764fb092fc498fd386dec96bc1277c0\": rpc error: code = NotFound desc = could not find container \"438a63a8ebb03df69558a54eb5bbd1c5d764fb092fc498fd386dec96bc1277c0\": container with ID starting with 438a63a8ebb03df69558a54eb5bbd1c5d764fb092fc498fd386dec96bc1277c0 not found: ID does not exist"
Sep 09 16:15:02.349646 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.349570    2567 scope.go:117] "RemoveContainer" containerID="f801b58f4e4fbc7dcb86f49a37ee64e4fd4492072a17df4f3ca1f09b44ac5830"
Sep 09 16:15:02.349801 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.349786    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f801b58f4e4fbc7dcb86f49a37ee64e4fd4492072a17df4f3ca1f09b44ac5830"} err="failed to get container status \"f801b58f4e4fbc7dcb86f49a37ee64e4fd4492072a17df4f3ca1f09b44ac5830\": rpc error: code = NotFound desc = could not find container \"f801b58f4e4fbc7dcb86f49a37ee64e4fd4492072a17df4f3ca1f09b44ac5830\": container with ID starting with f801b58f4e4fbc7dcb86f49a37ee64e4fd4492072a17df4f3ca1f09b44ac5830 not found: ID does not exist"
Sep 09 16:15:02.349846 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.349801    2567 scope.go:117] "RemoveContainer" containerID="7f94864c04e1832d8f0c1811ec85741834cc999412cb6bcbfaca226f0bea5b75"
Sep 09 16:15:02.350028 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.350001    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f94864c04e1832d8f0c1811ec85741834cc999412cb6bcbfaca226f0bea5b75"} err="failed to get container status \"7f94864c04e1832d8f0c1811ec85741834cc999412cb6bcbfaca226f0bea5b75\": rpc error: code = NotFound desc = could not find container \"7f94864c04e1832d8f0c1811ec85741834cc999412cb6bcbfaca226f0bea5b75\": container with ID starting with 7f94864c04e1832d8f0c1811ec85741834cc999412cb6bcbfaca226f0bea5b75 not found: ID does not exist"
Sep 09 16:15:02.350091 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.350030    2567 scope.go:117] "RemoveContainer" containerID="2bffbadca619f0bee3d548669ab94af54128e45c8273aea9620d6f92024a1a20"
Sep 09 16:15:02.350320 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.350301    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2bffbadca619f0bee3d548669ab94af54128e45c8273aea9620d6f92024a1a20"} err="failed to get container status \"2bffbadca619f0bee3d548669ab94af54128e45c8273aea9620d6f92024a1a20\": rpc error: code = NotFound desc = could not find container \"2bffbadca619f0bee3d548669ab94af54128e45c8273aea9620d6f92024a1a20\": container with ID starting with 2bffbadca619f0bee3d548669ab94af54128e45c8273aea9620d6f92024a1a20 not found: ID does not exist"
Sep 09 16:15:02.350320 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.350319    2567 scope.go:117] "RemoveContainer" containerID="4367e3d7420276b074cb71caae8ae96025e5e2134b741ba6b51ea812401cd3d5"
Sep 09 16:15:02.350606 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.350586    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4367e3d7420276b074cb71caae8ae96025e5e2134b741ba6b51ea812401cd3d5"} err="failed to get container status \"4367e3d7420276b074cb71caae8ae96025e5e2134b741ba6b51ea812401cd3d5\": rpc error: code = NotFound desc = could not find container \"4367e3d7420276b074cb71caae8ae96025e5e2134b741ba6b51ea812401cd3d5\": container with ID starting with 4367e3d7420276b074cb71caae8ae96025e5e2134b741ba6b51ea812401cd3d5 not found: ID does not exist"
Sep 09 16:15:02.350668 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.350607    2567 scope.go:117] "RemoveContainer" containerID="8cc86096bc9d62618a2c3de6f3f23cc3f129848857e335f6dee03c929c1a91b0"
Sep 09 16:15:02.350856 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.350835    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8cc86096bc9d62618a2c3de6f3f23cc3f129848857e335f6dee03c929c1a91b0"} err="failed to get container status \"8cc86096bc9d62618a2c3de6f3f23cc3f129848857e335f6dee03c929c1a91b0\": rpc error: code = NotFound desc = could not find container \"8cc86096bc9d62618a2c3de6f3f23cc3f129848857e335f6dee03c929c1a91b0\": container with ID starting with 8cc86096bc9d62618a2c3de6f3f23cc3f129848857e335f6dee03c929c1a91b0 not found: ID does not exist"
Sep 09 16:15:02.350932 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.350858    2567 scope.go:117] "RemoveContainer" containerID="29a120f9c840533949d3cbdeffc62e8b033c67aed2f1d0c8e2923effd6c83f4c"
Sep 09 16:15:02.351121 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.351103    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29a120f9c840533949d3cbdeffc62e8b033c67aed2f1d0c8e2923effd6c83f4c"} err="failed to get container status \"29a120f9c840533949d3cbdeffc62e8b033c67aed2f1d0c8e2923effd6c83f4c\": rpc error: code = NotFound desc = could not find container \"29a120f9c840533949d3cbdeffc62e8b033c67aed2f1d0c8e2923effd6c83f4c\": container with ID starting with 29a120f9c840533949d3cbdeffc62e8b033c67aed2f1d0c8e2923effd6c83f4c not found: ID does not exist"
Sep 09 16:15:02.351173 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.351123    2567 scope.go:117] "RemoveContainer" containerID="438a63a8ebb03df69558a54eb5bbd1c5d764fb092fc498fd386dec96bc1277c0"
Sep 09 16:15:02.351341 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.351323    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"438a63a8ebb03df69558a54eb5bbd1c5d764fb092fc498fd386dec96bc1277c0"} err="failed to get container status \"438a63a8ebb03df69558a54eb5bbd1c5d764fb092fc498fd386dec96bc1277c0\": rpc error: code = NotFound desc = could not find container \"438a63a8ebb03df69558a54eb5bbd1c5d764fb092fc498fd386dec96bc1277c0\": container with ID starting with 438a63a8ebb03df69558a54eb5bbd1c5d764fb092fc498fd386dec96bc1277c0 not found: ID does not exist"
Sep 09 16:15:02.351420 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.351344    2567 scope.go:117] "RemoveContainer" containerID="f801b58f4e4fbc7dcb86f49a37ee64e4fd4492072a17df4f3ca1f09b44ac5830"
Sep 09 16:15:02.351634 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.351617    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f801b58f4e4fbc7dcb86f49a37ee64e4fd4492072a17df4f3ca1f09b44ac5830"} err="failed to get container status \"f801b58f4e4fbc7dcb86f49a37ee64e4fd4492072a17df4f3ca1f09b44ac5830\": rpc error: code = NotFound desc = could not find container \"f801b58f4e4fbc7dcb86f49a37ee64e4fd4492072a17df4f3ca1f09b44ac5830\": container with ID starting with f801b58f4e4fbc7dcb86f49a37ee64e4fd4492072a17df4f3ca1f09b44ac5830 not found: ID does not exist"
Sep 09 16:15:02.351696 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.351635    2567 scope.go:117] "RemoveContainer" containerID="7f94864c04e1832d8f0c1811ec85741834cc999412cb6bcbfaca226f0bea5b75"
Sep 09 16:15:02.351854 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.351837    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f94864c04e1832d8f0c1811ec85741834cc999412cb6bcbfaca226f0bea5b75"} err="failed to get container status \"7f94864c04e1832d8f0c1811ec85741834cc999412cb6bcbfaca226f0bea5b75\": rpc error: code = NotFound desc = could not find container \"7f94864c04e1832d8f0c1811ec85741834cc999412cb6bcbfaca226f0bea5b75\": container with ID starting with 7f94864c04e1832d8f0c1811ec85741834cc999412cb6bcbfaca226f0bea5b75 not found: ID does not exist"
Sep 09 16:15:02.351905 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.351855    2567 scope.go:117] "RemoveContainer" containerID="2bffbadca619f0bee3d548669ab94af54128e45c8273aea9620d6f92024a1a20"
Sep 09 16:15:02.352088 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.352072    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2bffbadca619f0bee3d548669ab94af54128e45c8273aea9620d6f92024a1a20"} err="failed to get container status \"2bffbadca619f0bee3d548669ab94af54128e45c8273aea9620d6f92024a1a20\": rpc error: code = NotFound desc = could not find container \"2bffbadca619f0bee3d548669ab94af54128e45c8273aea9620d6f92024a1a20\": container with ID starting with 2bffbadca619f0bee3d548669ab94af54128e45c8273aea9620d6f92024a1a20 not found: ID does not exist"
Sep 09 16:15:02.352147 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.352088    2567 scope.go:117] "RemoveContainer" containerID="4367e3d7420276b074cb71caae8ae96025e5e2134b741ba6b51ea812401cd3d5"
Sep 09 16:15:02.352363 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.352345    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4367e3d7420276b074cb71caae8ae96025e5e2134b741ba6b51ea812401cd3d5"} err="failed to get container status \"4367e3d7420276b074cb71caae8ae96025e5e2134b741ba6b51ea812401cd3d5\": rpc error: code = NotFound desc = could not find container \"4367e3d7420276b074cb71caae8ae96025e5e2134b741ba6b51ea812401cd3d5\": container with ID starting with 4367e3d7420276b074cb71caae8ae96025e5e2134b741ba6b51ea812401cd3d5 not found: ID does not exist"
Sep 09 16:15:02.352443 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.352365    2567 scope.go:117] "RemoveContainer" containerID="8cc86096bc9d62618a2c3de6f3f23cc3f129848857e335f6dee03c929c1a91b0"
Sep 09 16:15:02.352629 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.352612    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8cc86096bc9d62618a2c3de6f3f23cc3f129848857e335f6dee03c929c1a91b0"} err="failed to get container status \"8cc86096bc9d62618a2c3de6f3f23cc3f129848857e335f6dee03c929c1a91b0\": rpc error: code = NotFound desc = could not find container \"8cc86096bc9d62618a2c3de6f3f23cc3f129848857e335f6dee03c929c1a91b0\": container with ID starting with 8cc86096bc9d62618a2c3de6f3f23cc3f129848857e335f6dee03c929c1a91b0 not found: ID does not exist"
Sep 09 16:15:02.352629 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.352628    2567 scope.go:117] "RemoveContainer" containerID="29a120f9c840533949d3cbdeffc62e8b033c67aed2f1d0c8e2923effd6c83f4c"
Sep 09 16:15:02.352866 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.352847    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29a120f9c840533949d3cbdeffc62e8b033c67aed2f1d0c8e2923effd6c83f4c"} err="failed to get container status \"29a120f9c840533949d3cbdeffc62e8b033c67aed2f1d0c8e2923effd6c83f4c\": rpc error: code = NotFound desc = could not find container \"29a120f9c840533949d3cbdeffc62e8b033c67aed2f1d0c8e2923effd6c83f4c\": container with ID starting with 29a120f9c840533949d3cbdeffc62e8b033c67aed2f1d0c8e2923effd6c83f4c not found: ID does not exist"
Sep 09 16:15:02.352866 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.352865    2567 scope.go:117] "RemoveContainer" containerID="438a63a8ebb03df69558a54eb5bbd1c5d764fb092fc498fd386dec96bc1277c0"
Sep 09 16:15:02.353093 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.353068    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"438a63a8ebb03df69558a54eb5bbd1c5d764fb092fc498fd386dec96bc1277c0"} err="failed to get container status \"438a63a8ebb03df69558a54eb5bbd1c5d764fb092fc498fd386dec96bc1277c0\": rpc error: code = NotFound desc = could not find container \"438a63a8ebb03df69558a54eb5bbd1c5d764fb092fc498fd386dec96bc1277c0\": container with ID starting with 438a63a8ebb03df69558a54eb5bbd1c5d764fb092fc498fd386dec96bc1277c0 not found: ID does not exist"
Sep 09 16:15:02.353171 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.353094    2567 scope.go:117] "RemoveContainer" containerID="f801b58f4e4fbc7dcb86f49a37ee64e4fd4492072a17df4f3ca1f09b44ac5830"
Sep 09 16:15:02.353358 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.353335    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f801b58f4e4fbc7dcb86f49a37ee64e4fd4492072a17df4f3ca1f09b44ac5830"} err="failed to get container status \"f801b58f4e4fbc7dcb86f49a37ee64e4fd4492072a17df4f3ca1f09b44ac5830\": rpc error: code = NotFound desc = could not find container \"f801b58f4e4fbc7dcb86f49a37ee64e4fd4492072a17df4f3ca1f09b44ac5830\": container with ID starting with f801b58f4e4fbc7dcb86f49a37ee64e4fd4492072a17df4f3ca1f09b44ac5830 not found: ID does not exist"
Sep 09 16:15:02.353429 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.353359    2567 scope.go:117] "RemoveContainer" containerID="7f94864c04e1832d8f0c1811ec85741834cc999412cb6bcbfaca226f0bea5b75"
Sep 09 16:15:02.353632 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.353613    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f94864c04e1832d8f0c1811ec85741834cc999412cb6bcbfaca226f0bea5b75"} err="failed to get container status \"7f94864c04e1832d8f0c1811ec85741834cc999412cb6bcbfaca226f0bea5b75\": rpc error: code = NotFound desc = could not find container \"7f94864c04e1832d8f0c1811ec85741834cc999412cb6bcbfaca226f0bea5b75\": container with ID starting with 7f94864c04e1832d8f0c1811ec85741834cc999412cb6bcbfaca226f0bea5b75 not found: ID does not exist"
Sep 09 16:15:02.353673 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.353633    2567 scope.go:117] "RemoveContainer" containerID="2bffbadca619f0bee3d548669ab94af54128e45c8273aea9620d6f92024a1a20"
Sep 09 16:15:02.353810 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.353793    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2bffbadca619f0bee3d548669ab94af54128e45c8273aea9620d6f92024a1a20"} err="failed to get container status \"2bffbadca619f0bee3d548669ab94af54128e45c8273aea9620d6f92024a1a20\": rpc error: code = NotFound desc = could not find container \"2bffbadca619f0bee3d548669ab94af54128e45c8273aea9620d6f92024a1a20\": container with ID starting with 2bffbadca619f0bee3d548669ab94af54128e45c8273aea9620d6f92024a1a20 not found: ID does not exist"
Sep 09 16:15:02.353865 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.353811    2567 scope.go:117] "RemoveContainer" containerID="4367e3d7420276b074cb71caae8ae96025e5e2134b741ba6b51ea812401cd3d5"
Sep 09 16:15:02.354055 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.354041    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4367e3d7420276b074cb71caae8ae96025e5e2134b741ba6b51ea812401cd3d5"} err="failed to get container status \"4367e3d7420276b074cb71caae8ae96025e5e2134b741ba6b51ea812401cd3d5\": rpc error: code = NotFound desc = could not find container \"4367e3d7420276b074cb71caae8ae96025e5e2134b741ba6b51ea812401cd3d5\": container with ID starting with 4367e3d7420276b074cb71caae8ae96025e5e2134b741ba6b51ea812401cd3d5 not found: ID does not exist"
Sep 09 16:15:02.354091 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.354056    2567 scope.go:117] "RemoveContainer" containerID="8cc86096bc9d62618a2c3de6f3f23cc3f129848857e335f6dee03c929c1a91b0"
Sep 09 16:15:02.354276 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.354252    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8cc86096bc9d62618a2c3de6f3f23cc3f129848857e335f6dee03c929c1a91b0"} err="failed to get container status \"8cc86096bc9d62618a2c3de6f3f23cc3f129848857e335f6dee03c929c1a91b0\": rpc error: code = NotFound desc = could not find container \"8cc86096bc9d62618a2c3de6f3f23cc3f129848857e335f6dee03c929c1a91b0\": container with ID starting with 8cc86096bc9d62618a2c3de6f3f23cc3f129848857e335f6dee03c929c1a91b0 not found: ID does not exist"
Sep 09 16:15:02.354343 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.354277    2567 scope.go:117] "RemoveContainer" containerID="29a120f9c840533949d3cbdeffc62e8b033c67aed2f1d0c8e2923effd6c83f4c"
Sep 09 16:15:02.354553 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.354537    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29a120f9c840533949d3cbdeffc62e8b033c67aed2f1d0c8e2923effd6c83f4c"} err="failed to get container status \"29a120f9c840533949d3cbdeffc62e8b033c67aed2f1d0c8e2923effd6c83f4c\": rpc error: code = NotFound desc = could not find container \"29a120f9c840533949d3cbdeffc62e8b033c67aed2f1d0c8e2923effd6c83f4c\": container with ID starting with 29a120f9c840533949d3cbdeffc62e8b033c67aed2f1d0c8e2923effd6c83f4c not found: ID does not exist"
Sep 09 16:15:02.354610 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.354553    2567 scope.go:117] "RemoveContainer" containerID="438a63a8ebb03df69558a54eb5bbd1c5d764fb092fc498fd386dec96bc1277c0"
Sep 09 16:15:02.354764 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.354747    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"438a63a8ebb03df69558a54eb5bbd1c5d764fb092fc498fd386dec96bc1277c0"} err="failed to get container status \"438a63a8ebb03df69558a54eb5bbd1c5d764fb092fc498fd386dec96bc1277c0\": rpc error: code = NotFound desc = could not find container \"438a63a8ebb03df69558a54eb5bbd1c5d764fb092fc498fd386dec96bc1277c0\": container with ID starting with 438a63a8ebb03df69558a54eb5bbd1c5d764fb092fc498fd386dec96bc1277c0 not found: ID does not exist"
Sep 09 16:15:02.354826 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.354767    2567 scope.go:117] "RemoveContainer" containerID="f801b58f4e4fbc7dcb86f49a37ee64e4fd4492072a17df4f3ca1f09b44ac5830"
Sep 09 16:15:02.354987 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.354971    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f801b58f4e4fbc7dcb86f49a37ee64e4fd4492072a17df4f3ca1f09b44ac5830"} err="failed to get container status \"f801b58f4e4fbc7dcb86f49a37ee64e4fd4492072a17df4f3ca1f09b44ac5830\": rpc error: code = NotFound desc = could not find container \"f801b58f4e4fbc7dcb86f49a37ee64e4fd4492072a17df4f3ca1f09b44ac5830\": container with ID starting with f801b58f4e4fbc7dcb86f49a37ee64e4fd4492072a17df4f3ca1f09b44ac5830 not found: ID does not exist"
Sep 09 16:15:02.355050 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.354989    2567 scope.go:117] "RemoveContainer" containerID="7f94864c04e1832d8f0c1811ec85741834cc999412cb6bcbfaca226f0bea5b75"
Sep 09 16:15:02.355196 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.355181    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f94864c04e1832d8f0c1811ec85741834cc999412cb6bcbfaca226f0bea5b75"} err="failed to get container status \"7f94864c04e1832d8f0c1811ec85741834cc999412cb6bcbfaca226f0bea5b75\": rpc error: code = NotFound desc = could not find container \"7f94864c04e1832d8f0c1811ec85741834cc999412cb6bcbfaca226f0bea5b75\": container with ID starting with 7f94864c04e1832d8f0c1811ec85741834cc999412cb6bcbfaca226f0bea5b75 not found: ID does not exist"
Sep 09 16:15:02.355235 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.355196    2567 scope.go:117] "RemoveContainer" containerID="2bffbadca619f0bee3d548669ab94af54128e45c8273aea9620d6f92024a1a20"
Sep 09 16:15:02.355431 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.355414    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2bffbadca619f0bee3d548669ab94af54128e45c8273aea9620d6f92024a1a20"} err="failed to get container status \"2bffbadca619f0bee3d548669ab94af54128e45c8273aea9620d6f92024a1a20\": rpc error: code = NotFound desc = could not find container \"2bffbadca619f0bee3d548669ab94af54128e45c8273aea9620d6f92024a1a20\": container with ID starting with 2bffbadca619f0bee3d548669ab94af54128e45c8273aea9620d6f92024a1a20 not found: ID does not exist"
Sep 09 16:15:02.355431 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.355429    2567 scope.go:117] "RemoveContainer" containerID="4367e3d7420276b074cb71caae8ae96025e5e2134b741ba6b51ea812401cd3d5"
Sep 09 16:15:02.355653 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.355627    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4367e3d7420276b074cb71caae8ae96025e5e2134b741ba6b51ea812401cd3d5"} err="failed to get container status \"4367e3d7420276b074cb71caae8ae96025e5e2134b741ba6b51ea812401cd3d5\": rpc error: code = NotFound desc = could not find container \"4367e3d7420276b074cb71caae8ae96025e5e2134b741ba6b51ea812401cd3d5\": container with ID starting with 4367e3d7420276b074cb71caae8ae96025e5e2134b741ba6b51ea812401cd3d5 not found: ID does not exist"
Sep 09 16:15:02.359043 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.359018    2567 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/84e0c89a-ea89-4303-ab75-20cd7ddf0752-config\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:15:02.359158 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.359050    2567 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/84e0c89a-ea89-4303-ab75-20cd7ddf0752-web-config\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:15:02.359158 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.359067    2567 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/84e0c89a-ea89-4303-ab75-20cd7ddf0752-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:15:02.359158 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.359080    2567 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84e0c89a-ea89-4303-ab75-20cd7ddf0752-prometheus-trusted-ca-bundle\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:15:02.359158 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.359094    2567 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/84e0c89a-ea89-4303-ab75-20cd7ddf0752-secret-grpc-tls\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:15:02.359158 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.359104    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nw9s6\" (UniqueName: \"kubernetes.io/projected/84e0c89a-ea89-4303-ab75-20cd7ddf0752-kube-api-access-nw9s6\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:15:02.359158 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.359113    2567 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/84e0c89a-ea89-4303-ab75-20cd7ddf0752-thanos-prometheus-http-client-file\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:15:02.359158 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.359122    2567 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/84e0c89a-ea89-4303-ab75-20cd7ddf0752-configmap-metrics-client-ca\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:15:02.359158 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.359131    2567 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/84e0c89a-ea89-4303-ab75-20cd7ddf0752-secret-prometheus-k8s-tls\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:15:02.359158 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.359139    2567 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/84e0c89a-ea89-4303-ab75-20cd7ddf0752-prometheus-k8s-db\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:15:02.359158 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.359147    2567 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/84e0c89a-ea89-4303-ab75-20cd7ddf0752-config-out\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:15:02.359158 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.359155    2567 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/84e0c89a-ea89-4303-ab75-20cd7ddf0752-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:15:02.359158 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.359162    2567 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/84e0c89a-ea89-4303-ab75-20cd7ddf0752-tls-assets\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:15:02.359535 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.359170    2567 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/84e0c89a-ea89-4303-ab75-20cd7ddf0752-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:15:02.359535 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.359178    2567 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/84e0c89a-ea89-4303-ab75-20cd7ddf0752-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:15:02.359535 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.359186    2567 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/84e0c89a-ea89-4303-ab75-20cd7ddf0752-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:15:02.359535 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.359194    2567 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/84e0c89a-ea89-4303-ab75-20cd7ddf0752-secret-kube-rbac-proxy\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:15:02.359535 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.359202    2567 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/84e0c89a-ea89-4303-ab75-20cd7ddf0752-secret-metrics-client-certs\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:15:02.603102 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.603062    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 09 16:15:02.609787 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.609763    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 09 16:15:02.634930 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.634902    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 09 16:15:02.635182 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.635171    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84e0c89a-ea89-4303-ab75-20cd7ddf0752" containerName="kube-rbac-proxy"
Sep 09 16:15:02.635224 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.635185    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="84e0c89a-ea89-4303-ab75-20cd7ddf0752" containerName="kube-rbac-proxy"
Sep 09 16:15:02.635224 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.635196    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84e0c89a-ea89-4303-ab75-20cd7ddf0752" containerName="init-config-reloader"
Sep 09 16:15:02.635224 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.635202    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="84e0c89a-ea89-4303-ab75-20cd7ddf0752" containerName="init-config-reloader"
Sep 09 16:15:02.635224 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.635209    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="071d80b2-9f57-45c3-980d-678dc98590f1" containerName="registry"
Sep 09 16:15:02.635224 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.635214    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="071d80b2-9f57-45c3-980d-678dc98590f1" containerName="registry"
Sep 09 16:15:02.635224 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.635226    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84e0c89a-ea89-4303-ab75-20cd7ddf0752" containerName="prometheus"
Sep 09 16:15:02.635385 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.635231    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="84e0c89a-ea89-4303-ab75-20cd7ddf0752" containerName="prometheus"
Sep 09 16:15:02.635385 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.635239    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84e0c89a-ea89-4303-ab75-20cd7ddf0752" containerName="kube-rbac-proxy-web"
Sep 09 16:15:02.635385 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.635244    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="84e0c89a-ea89-4303-ab75-20cd7ddf0752" containerName="kube-rbac-proxy-web"
Sep 09 16:15:02.635385 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.635250    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84e0c89a-ea89-4303-ab75-20cd7ddf0752" containerName="kube-rbac-proxy-thanos"
Sep 09 16:15:02.635385 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.635255    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="84e0c89a-ea89-4303-ab75-20cd7ddf0752" containerName="kube-rbac-proxy-thanos"
Sep 09 16:15:02.635385 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.635263    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84e0c89a-ea89-4303-ab75-20cd7ddf0752" containerName="config-reloader"
Sep 09 16:15:02.635385 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.635268    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="84e0c89a-ea89-4303-ab75-20cd7ddf0752" containerName="config-reloader"
Sep 09 16:15:02.635385 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.635275    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84e0c89a-ea89-4303-ab75-20cd7ddf0752" containerName="thanos-sidecar"
Sep 09 16:15:02.635385 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.635279    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="84e0c89a-ea89-4303-ab75-20cd7ddf0752" containerName="thanos-sidecar"
Sep 09 16:15:02.635385 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.635319    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="071d80b2-9f57-45c3-980d-678dc98590f1" containerName="registry"
Sep 09 16:15:02.635385 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.635326    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="84e0c89a-ea89-4303-ab75-20cd7ddf0752" containerName="config-reloader"
Sep 09 16:15:02.635385 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.635332    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="84e0c89a-ea89-4303-ab75-20cd7ddf0752" containerName="thanos-sidecar"
Sep 09 16:15:02.635385 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.635337    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="84e0c89a-ea89-4303-ab75-20cd7ddf0752" containerName="kube-rbac-proxy-web"
Sep 09 16:15:02.635385 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.635342    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="84e0c89a-ea89-4303-ab75-20cd7ddf0752" containerName="kube-rbac-proxy"
Sep 09 16:15:02.635385 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.635346    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="84e0c89a-ea89-4303-ab75-20cd7ddf0752" containerName="prometheus"
Sep 09 16:15:02.635385 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.635350    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="84e0c89a-ea89-4303-ab75-20cd7ddf0752" containerName="kube-rbac-proxy-thanos"
Sep 09 16:15:02.640433 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.640412    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:15:02.643160 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.643118    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 09 16:15:02.643160 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.643128    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 09 16:15:02.643339 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.643192    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 09 16:15:02.643339 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.643199    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 09 16:15:02.643339 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.643199    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 09 16:15:02.643504 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.643491    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-gdp56\""
Sep 09 16:15:02.643891 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.643855    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 09 16:15:02.643891 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.643874    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 09 16:15:02.644025 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.643900    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 09 16:15:02.644025 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.643882    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-3hrttl78b20d8\""
Sep 09 16:15:02.644025 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.643989    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 09 16:15:02.644179 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.644166    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 09 16:15:02.644255 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.644242    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 09 16:15:02.644481 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.644446    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 09 16:15:02.647007 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.646984    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 09 16:15:02.648764 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.648745    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 09 16:15:02.654205 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.654183    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 09 16:15:02.661052 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.661030    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fefe2c92-8751-43f2-b904-d80d04c9340a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"fefe2c92-8751-43f2-b904-d80d04c9340a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:15:02.661150 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.661062    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fefe2c92-8751-43f2-b904-d80d04c9340a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"fefe2c92-8751-43f2-b904-d80d04c9340a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:15:02.661150 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.661103    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fefe2c92-8751-43f2-b904-d80d04c9340a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fefe2c92-8751-43f2-b904-d80d04c9340a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:15:02.661150 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.661129    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/fefe2c92-8751-43f2-b904-d80d04c9340a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"fefe2c92-8751-43f2-b904-d80d04c9340a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:15:02.661326 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.661154    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/fefe2c92-8751-43f2-b904-d80d04c9340a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"fefe2c92-8751-43f2-b904-d80d04c9340a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:15:02.661326 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.661193    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fefe2c92-8751-43f2-b904-d80d04c9340a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fefe2c92-8751-43f2-b904-d80d04c9340a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:15:02.661326 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.661249    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrwm4\" (UniqueName: \"kubernetes.io/projected/fefe2c92-8751-43f2-b904-d80d04c9340a-kube-api-access-wrwm4\") pod \"prometheus-k8s-0\" (UID: \"fefe2c92-8751-43f2-b904-d80d04c9340a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:15:02.661326 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.661289    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/fefe2c92-8751-43f2-b904-d80d04c9340a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"fefe2c92-8751-43f2-b904-d80d04c9340a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:15:02.661565 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.661355    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/fefe2c92-8751-43f2-b904-d80d04c9340a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"fefe2c92-8751-43f2-b904-d80d04c9340a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:15:02.661565 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.661382    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/fefe2c92-8751-43f2-b904-d80d04c9340a-config\") pod \"prometheus-k8s-0\" (UID: \"fefe2c92-8751-43f2-b904-d80d04c9340a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:15:02.661565 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.661400    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fefe2c92-8751-43f2-b904-d80d04c9340a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fefe2c92-8751-43f2-b904-d80d04c9340a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:15:02.661565 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.661416    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fefe2c92-8751-43f2-b904-d80d04c9340a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"fefe2c92-8751-43f2-b904-d80d04c9340a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:15:02.661565 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.661437    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/fefe2c92-8751-43f2-b904-d80d04c9340a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"fefe2c92-8751-43f2-b904-d80d04c9340a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:15:02.661565 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.661510    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fefe2c92-8751-43f2-b904-d80d04c9340a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"fefe2c92-8751-43f2-b904-d80d04c9340a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:15:02.661565 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.661529    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fefe2c92-8751-43f2-b904-d80d04c9340a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"fefe2c92-8751-43f2-b904-d80d04c9340a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:15:02.661565 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.661549    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fefe2c92-8751-43f2-b904-d80d04c9340a-config-out\") pod \"prometheus-k8s-0\" (UID: \"fefe2c92-8751-43f2-b904-d80d04c9340a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:15:02.661833 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.661608    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/fefe2c92-8751-43f2-b904-d80d04c9340a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"fefe2c92-8751-43f2-b904-d80d04c9340a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:15:02.661833 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.661643    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/fefe2c92-8751-43f2-b904-d80d04c9340a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"fefe2c92-8751-43f2-b904-d80d04c9340a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:15:02.661833 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.661671    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fefe2c92-8751-43f2-b904-d80d04c9340a-web-config\") pod \"prometheus-k8s-0\" (UID: \"fefe2c92-8751-43f2-b904-d80d04c9340a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:15:02.661833 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.661696    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/fefe2c92-8751-43f2-b904-d80d04c9340a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"fefe2c92-8751-43f2-b904-d80d04c9340a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:15:02.762354 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.762320    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fefe2c92-8751-43f2-b904-d80d04c9340a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fefe2c92-8751-43f2-b904-d80d04c9340a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:15:02.762562 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.762365    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrwm4\" (UniqueName: \"kubernetes.io/projected/fefe2c92-8751-43f2-b904-d80d04c9340a-kube-api-access-wrwm4\") pod \"prometheus-k8s-0\" (UID: \"fefe2c92-8751-43f2-b904-d80d04c9340a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:15:02.762607 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.762584    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/fefe2c92-8751-43f2-b904-d80d04c9340a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"fefe2c92-8751-43f2-b904-d80d04c9340a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:15:02.762641 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.762632    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/fefe2c92-8751-43f2-b904-d80d04c9340a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"fefe2c92-8751-43f2-b904-d80d04c9340a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:15:02.762695 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.762664    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/fefe2c92-8751-43f2-b904-d80d04c9340a-config\") pod \"prometheus-k8s-0\" (UID: \"fefe2c92-8751-43f2-b904-d80d04c9340a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:15:02.762728 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.762712    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fefe2c92-8751-43f2-b904-d80d04c9340a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fefe2c92-8751-43f2-b904-d80d04c9340a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:15:02.762765 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.762750    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fefe2c92-8751-43f2-b904-d80d04c9340a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"fefe2c92-8751-43f2-b904-d80d04c9340a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:15:02.762813 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.762800    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/fefe2c92-8751-43f2-b904-d80d04c9340a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"fefe2c92-8751-43f2-b904-d80d04c9340a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:15:02.762858 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.762832    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fefe2c92-8751-43f2-b904-d80d04c9340a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"fefe2c92-8751-43f2-b904-d80d04c9340a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:15:02.762912 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.762857    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fefe2c92-8751-43f2-b904-d80d04c9340a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"fefe2c92-8751-43f2-b904-d80d04c9340a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:15:02.762912 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.762891    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fefe2c92-8751-43f2-b904-d80d04c9340a-config-out\") pod \"prometheus-k8s-0\" (UID: \"fefe2c92-8751-43f2-b904-d80d04c9340a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:15:02.763002 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.762928    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/fefe2c92-8751-43f2-b904-d80d04c9340a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"fefe2c92-8751-43f2-b904-d80d04c9340a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:15:02.763002 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.762954    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/fefe2c92-8751-43f2-b904-d80d04c9340a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"fefe2c92-8751-43f2-b904-d80d04c9340a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:15:02.763002 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.762998    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fefe2c92-8751-43f2-b904-d80d04c9340a-web-config\") pod \"prometheus-k8s-0\" (UID: \"fefe2c92-8751-43f2-b904-d80d04c9340a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:15:02.763140 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.763026    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/fefe2c92-8751-43f2-b904-d80d04c9340a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"fefe2c92-8751-43f2-b904-d80d04c9340a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:15:02.763140 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.763049    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fefe2c92-8751-43f2-b904-d80d04c9340a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"fefe2c92-8751-43f2-b904-d80d04c9340a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:15:02.763140 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.763074    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fefe2c92-8751-43f2-b904-d80d04c9340a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"fefe2c92-8751-43f2-b904-d80d04c9340a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:15:02.763140 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.763129    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fefe2c92-8751-43f2-b904-d80d04c9340a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fefe2c92-8751-43f2-b904-d80d04c9340a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:15:02.763326 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.763157    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/fefe2c92-8751-43f2-b904-d80d04c9340a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"fefe2c92-8751-43f2-b904-d80d04c9340a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:15:02.763326 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.763181    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/fefe2c92-8751-43f2-b904-d80d04c9340a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"fefe2c92-8751-43f2-b904-d80d04c9340a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:15:02.763326 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.763242    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fefe2c92-8751-43f2-b904-d80d04c9340a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fefe2c92-8751-43f2-b904-d80d04c9340a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:15:02.763511 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.763319    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/fefe2c92-8751-43f2-b904-d80d04c9340a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"fefe2c92-8751-43f2-b904-d80d04c9340a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:15:02.763807 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.763726    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/fefe2c92-8751-43f2-b904-d80d04c9340a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"fefe2c92-8751-43f2-b904-d80d04c9340a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:15:02.763807 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.763820    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/fefe2c92-8751-43f2-b904-d80d04c9340a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"fefe2c92-8751-43f2-b904-d80d04c9340a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:15:02.764021 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.763862    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fefe2c92-8751-43f2-b904-d80d04c9340a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fefe2c92-8751-43f2-b904-d80d04c9340a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:15:02.764358 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.764336    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fefe2c92-8751-43f2-b904-d80d04c9340a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"fefe2c92-8751-43f2-b904-d80d04c9340a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:15:02.764748 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.764720    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fefe2c92-8751-43f2-b904-d80d04c9340a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fefe2c92-8751-43f2-b904-d80d04c9340a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:15:02.766240 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.765928    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/fefe2c92-8751-43f2-b904-d80d04c9340a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"fefe2c92-8751-43f2-b904-d80d04c9340a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:15:02.766240 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.766013    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fefe2c92-8751-43f2-b904-d80d04c9340a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"fefe2c92-8751-43f2-b904-d80d04c9340a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:15:02.766240 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.766022    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fefe2c92-8751-43f2-b904-d80d04c9340a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"fefe2c92-8751-43f2-b904-d80d04c9340a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:15:02.766526 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.766409    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fefe2c92-8751-43f2-b904-d80d04c9340a-config-out\") pod \"prometheus-k8s-0\" (UID: \"fefe2c92-8751-43f2-b904-d80d04c9340a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:15:02.766629 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.766601    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/fefe2c92-8751-43f2-b904-d80d04c9340a-config\") pod \"prometheus-k8s-0\" (UID: \"fefe2c92-8751-43f2-b904-d80d04c9340a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:15:02.766782 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.766761    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fefe2c92-8751-43f2-b904-d80d04c9340a-web-config\") pod \"prometheus-k8s-0\" (UID: \"fefe2c92-8751-43f2-b904-d80d04c9340a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:15:02.767535 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.767510    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/fefe2c92-8751-43f2-b904-d80d04c9340a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"fefe2c92-8751-43f2-b904-d80d04c9340a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:15:02.768157 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.768131    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/fefe2c92-8751-43f2-b904-d80d04c9340a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"fefe2c92-8751-43f2-b904-d80d04c9340a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:15:02.768476 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.768430    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/fefe2c92-8751-43f2-b904-d80d04c9340a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"fefe2c92-8751-43f2-b904-d80d04c9340a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:15:02.768571 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.768554    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/fefe2c92-8751-43f2-b904-d80d04c9340a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"fefe2c92-8751-43f2-b904-d80d04c9340a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:15:02.769114 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.769095    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fefe2c92-8751-43f2-b904-d80d04c9340a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"fefe2c92-8751-43f2-b904-d80d04c9340a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:15:02.769155 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.769138    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fefe2c92-8751-43f2-b904-d80d04c9340a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"fefe2c92-8751-43f2-b904-d80d04c9340a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:15:02.770268 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.770252    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrwm4\" (UniqueName: \"kubernetes.io/projected/fefe2c92-8751-43f2-b904-d80d04c9340a-kube-api-access-wrwm4\") pod \"prometheus-k8s-0\" (UID: \"fefe2c92-8751-43f2-b904-d80d04c9340a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:15:02.950804 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:02.950709    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:15:03.034950 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:03.034911    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84e0c89a-ea89-4303-ab75-20cd7ddf0752" path="/var/lib/kubelet/pods/84e0c89a-ea89-4303-ab75-20cd7ddf0752/volumes"
Sep 09 16:15:03.233061 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:03.232985    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 09 16:15:03.236134 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:15:03.236106    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfefe2c92_8751_43f2_b904_d80d04c9340a.slice/crio-f87c70531e88abdc839690ee5df8a3a32208ba9510055a45aae584b440ee4100 WatchSource:0}: Error finding container f87c70531e88abdc839690ee5df8a3a32208ba9510055a45aae584b440ee4100: Status 404 returned error can't find the container with id f87c70531e88abdc839690ee5df8a3a32208ba9510055a45aae584b440ee4100
Sep 09 16:15:03.284225 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:03.284193    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fefe2c92-8751-43f2-b904-d80d04c9340a","Type":"ContainerStarted","Data":"f87c70531e88abdc839690ee5df8a3a32208ba9510055a45aae584b440ee4100"}
Sep 09 16:15:04.297309 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:04.297215    2567 generic.go:356] "Generic (PLEG): container finished" podID="fefe2c92-8751-43f2-b904-d80d04c9340a" containerID="a1585931a69a4e25773f04c258d953612fb845d18b67cc60c65182f98a4b608a" exitCode=0
Sep 09 16:15:04.297309 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:04.297297    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fefe2c92-8751-43f2-b904-d80d04c9340a","Type":"ContainerDied","Data":"a1585931a69a4e25773f04c258d953612fb845d18b67cc60c65182f98a4b608a"}
Sep 09 16:15:05.304442 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:05.304402    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fefe2c92-8751-43f2-b904-d80d04c9340a","Type":"ContainerStarted","Data":"8adb08fcb498cd77283f588eb5844150d445a9483a3c4b9601e4eb682161577a"}
Sep 09 16:15:05.304830 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:05.304509    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fefe2c92-8751-43f2-b904-d80d04c9340a","Type":"ContainerStarted","Data":"5b11e9b67b3e30e925c8fa1e2d09525b2ccd595fe26324f04bfe4da181b649d6"}
Sep 09 16:15:05.304830 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:05.304550    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fefe2c92-8751-43f2-b904-d80d04c9340a","Type":"ContainerStarted","Data":"5eb6dd4f7a750e2995075041b9816d6d92d4c9a36e40aa7279c7d1a2723998f8"}
Sep 09 16:15:05.304830 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:05.304590    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fefe2c92-8751-43f2-b904-d80d04c9340a","Type":"ContainerStarted","Data":"d2cf284853fc703aa8ceb399a0649eb832f11962443348510daf2538e87f919d"}
Sep 09 16:15:05.304830 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:05.304626    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fefe2c92-8751-43f2-b904-d80d04c9340a","Type":"ContainerStarted","Data":"b0221f56b4e7ddff57ba1f12d288c0354a3e7f683d89f87d6693a8b250dedcc4"}
Sep 09 16:15:05.304830 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:05.304660    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fefe2c92-8751-43f2-b904-d80d04c9340a","Type":"ContainerStarted","Data":"0ec1cfb99eeb7fafb745e8255d17df13b0329c0b034fd3c1dfad048ddc5d12e0"}
Sep 09 16:15:05.333509 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:05.333430    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.311449529 podStartE2EDuration="3.333412693s" totalImagesPullingTime="0s" totalInitContainerRuntime="21.963164ms" isStatefulPod=false podCreationTimestamp="2026-09-09 16:15:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 16:15:05.331134298 +0000 UTC m=+380.748984331" watchObservedRunningTime="2026-09-09 16:15:05.333412693 +0000 UTC m=+380.751262691"
Sep 09 16:15:07.951504 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:15:07.951449    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:16:02.951092 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:16:02.951050    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:16:02.966953 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:16:02.966927    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:16:03.536310 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:16:03.536280    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 16:17:17.064467 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:17.064428    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7bc87557cd-6bpm8"]
Sep 09 16:17:17.067560 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:17.067539    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7bc87557cd-6bpm8"
Sep 09 16:17:17.070080 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:17.070061    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 09 16:17:17.070985 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:17.070966    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 09 16:17:17.071240 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:17.071219    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 09 16:17:17.071350 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:17.071237    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-dwhmp\""
Sep 09 16:17:17.071350 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:17.071306    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 09 16:17:17.071435 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:17.071358    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 09 16:17:17.079115 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:17.079096    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7bc87557cd-6bpm8"]
Sep 09 16:17:17.148015 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:17.147986    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/83e7f88c-6264-429c-a506-f98dce47d2f7-cert\") pod \"lws-controller-manager-7bc87557cd-6bpm8\" (UID: \"83e7f88c-6264-429c-a506-f98dce47d2f7\") " pod="openshift-lws-operator/lws-controller-manager-7bc87557cd-6bpm8"
Sep 09 16:17:17.148182 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:17.148083    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/83e7f88c-6264-429c-a506-f98dce47d2f7-manager-config\") pod \"lws-controller-manager-7bc87557cd-6bpm8\" (UID: \"83e7f88c-6264-429c-a506-f98dce47d2f7\") " pod="openshift-lws-operator/lws-controller-manager-7bc87557cd-6bpm8"
Sep 09 16:17:17.148182 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:17.148114    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/83e7f88c-6264-429c-a506-f98dce47d2f7-metrics-cert\") pod \"lws-controller-manager-7bc87557cd-6bpm8\" (UID: \"83e7f88c-6264-429c-a506-f98dce47d2f7\") " pod="openshift-lws-operator/lws-controller-manager-7bc87557cd-6bpm8"
Sep 09 16:17:17.148182 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:17.148143    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrh64\" (UniqueName: \"kubernetes.io/projected/83e7f88c-6264-429c-a506-f98dce47d2f7-kube-api-access-mrh64\") pod \"lws-controller-manager-7bc87557cd-6bpm8\" (UID: \"83e7f88c-6264-429c-a506-f98dce47d2f7\") " pod="openshift-lws-operator/lws-controller-manager-7bc87557cd-6bpm8"
Sep 09 16:17:17.248898 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:17.248863    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/83e7f88c-6264-429c-a506-f98dce47d2f7-cert\") pod \"lws-controller-manager-7bc87557cd-6bpm8\" (UID: \"83e7f88c-6264-429c-a506-f98dce47d2f7\") " pod="openshift-lws-operator/lws-controller-manager-7bc87557cd-6bpm8"
Sep 09 16:17:17.249069 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:17.248927    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/83e7f88c-6264-429c-a506-f98dce47d2f7-manager-config\") pod \"lws-controller-manager-7bc87557cd-6bpm8\" (UID: \"83e7f88c-6264-429c-a506-f98dce47d2f7\") " pod="openshift-lws-operator/lws-controller-manager-7bc87557cd-6bpm8"
Sep 09 16:17:17.249069 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:17.248946    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/83e7f88c-6264-429c-a506-f98dce47d2f7-metrics-cert\") pod \"lws-controller-manager-7bc87557cd-6bpm8\" (UID: \"83e7f88c-6264-429c-a506-f98dce47d2f7\") " pod="openshift-lws-operator/lws-controller-manager-7bc87557cd-6bpm8"
Sep 09 16:17:17.249069 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:17.248972    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrh64\" (UniqueName: \"kubernetes.io/projected/83e7f88c-6264-429c-a506-f98dce47d2f7-kube-api-access-mrh64\") pod \"lws-controller-manager-7bc87557cd-6bpm8\" (UID: \"83e7f88c-6264-429c-a506-f98dce47d2f7\") " pod="openshift-lws-operator/lws-controller-manager-7bc87557cd-6bpm8"
Sep 09 16:17:17.249698 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:17.249593    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/83e7f88c-6264-429c-a506-f98dce47d2f7-manager-config\") pod \"lws-controller-manager-7bc87557cd-6bpm8\" (UID: \"83e7f88c-6264-429c-a506-f98dce47d2f7\") " pod="openshift-lws-operator/lws-controller-manager-7bc87557cd-6bpm8"
Sep 09 16:17:17.251600 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:17.251577    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/83e7f88c-6264-429c-a506-f98dce47d2f7-cert\") pod \"lws-controller-manager-7bc87557cd-6bpm8\" (UID: \"83e7f88c-6264-429c-a506-f98dce47d2f7\") " pod="openshift-lws-operator/lws-controller-manager-7bc87557cd-6bpm8"
Sep 09 16:17:17.251714 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:17.251586    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/83e7f88c-6264-429c-a506-f98dce47d2f7-metrics-cert\") pod \"lws-controller-manager-7bc87557cd-6bpm8\" (UID: \"83e7f88c-6264-429c-a506-f98dce47d2f7\") " pod="openshift-lws-operator/lws-controller-manager-7bc87557cd-6bpm8"
Sep 09 16:17:17.257758 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:17.257738    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrh64\" (UniqueName: \"kubernetes.io/projected/83e7f88c-6264-429c-a506-f98dce47d2f7-kube-api-access-mrh64\") pod \"lws-controller-manager-7bc87557cd-6bpm8\" (UID: \"83e7f88c-6264-429c-a506-f98dce47d2f7\") " pod="openshift-lws-operator/lws-controller-manager-7bc87557cd-6bpm8"
Sep 09 16:17:17.376330 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:17.376287    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7bc87557cd-6bpm8"
Sep 09 16:17:17.665664 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:17.665637    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7bc87557cd-6bpm8"]
Sep 09 16:17:17.667617 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:17:17.667589    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83e7f88c_6264_429c_a506_f98dce47d2f7.slice/crio-a86421bf59f77c4cd30e9a98448cad83ac6d50f81abc999bc88183a614bae1f1 WatchSource:0}: Error finding container a86421bf59f77c4cd30e9a98448cad83ac6d50f81abc999bc88183a614bae1f1: Status 404 returned error can't find the container with id a86421bf59f77c4cd30e9a98448cad83ac6d50f81abc999bc88183a614bae1f1
Sep 09 16:17:17.797063 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:17.797029    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7bc87557cd-6bpm8" event={"ID":"83e7f88c-6264-429c-a506-f98dce47d2f7","Type":"ContainerStarted","Data":"a86421bf59f77c4cd30e9a98448cad83ac6d50f81abc999bc88183a614bae1f1"}
Sep 09 16:17:20.810558 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:20.810519    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7bc87557cd-6bpm8" event={"ID":"83e7f88c-6264-429c-a506-f98dce47d2f7","Type":"ContainerStarted","Data":"53514a99ee5f39b55e85f7a575fcdcbacbe6dba437d156d530660ba13f94f55e"}
Sep 09 16:17:20.810960 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:20.810762    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7bc87557cd-6bpm8"
Sep 09 16:17:20.840396 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:20.840299    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7bc87557cd-6bpm8" podStartSLOduration=0.932486816 podStartE2EDuration="3.840284713s" totalImagesPullingTime="2.907797897s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 16:17:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 16:17:20.839294773 +0000 UTC m=+516.257144772" watchObservedRunningTime="2026-09-09 16:17:20.840284713 +0000 UTC m=+516.258134711"
Sep 09 16:17:24.162965 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:24.162892    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-67c7bcf47f-h6rbf"]
Sep 09 16:17:24.167571 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:24.167546    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-67c7bcf47f-h6rbf"
Sep 09 16:17:24.174603 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:24.174576    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 09 16:17:24.174734 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:24.174635    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 09 16:17:24.174893 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:24.174877    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 09 16:17:24.175059 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:24.175039    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 09 16:17:24.175144 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:24.175057    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-vdnpc\""
Sep 09 16:17:24.205377 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:24.205339    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-67c7bcf47f-h6rbf"]
Sep 09 16:17:24.311350 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:24.311307    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c91ca7cb-43cc-439a-90e4-15c989659880-manifests\") pod \"opendatahub-operator-controller-manager-67c7bcf47f-h6rbf\" (UID: \"c91ca7cb-43cc-439a-90e4-15c989659880\") " pod="opendatahub/opendatahub-operator-controller-manager-67c7bcf47f-h6rbf"
Sep 09 16:17:24.311350 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:24.311355    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8zfs\" (UniqueName: \"kubernetes.io/projected/c91ca7cb-43cc-439a-90e4-15c989659880-kube-api-access-b8zfs\") pod \"opendatahub-operator-controller-manager-67c7bcf47f-h6rbf\" (UID: \"c91ca7cb-43cc-439a-90e4-15c989659880\") " pod="opendatahub/opendatahub-operator-controller-manager-67c7bcf47f-h6rbf"
Sep 09 16:17:24.311626 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:24.311376    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c91ca7cb-43cc-439a-90e4-15c989659880-tmp\") pod \"opendatahub-operator-controller-manager-67c7bcf47f-h6rbf\" (UID: \"c91ca7cb-43cc-439a-90e4-15c989659880\") " pod="opendatahub/opendatahub-operator-controller-manager-67c7bcf47f-h6rbf"
Sep 09 16:17:24.311626 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:24.311482    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c91ca7cb-43cc-439a-90e4-15c989659880-apiservice-cert\") pod \"opendatahub-operator-controller-manager-67c7bcf47f-h6rbf\" (UID: \"c91ca7cb-43cc-439a-90e4-15c989659880\") " pod="opendatahub/opendatahub-operator-controller-manager-67c7bcf47f-h6rbf"
Sep 09 16:17:24.311626 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:24.311515    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c91ca7cb-43cc-439a-90e4-15c989659880-webhook-cert\") pod \"opendatahub-operator-controller-manager-67c7bcf47f-h6rbf\" (UID: \"c91ca7cb-43cc-439a-90e4-15c989659880\") " pod="opendatahub/opendatahub-operator-controller-manager-67c7bcf47f-h6rbf"
Sep 09 16:17:24.412901 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:24.412858    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c91ca7cb-43cc-439a-90e4-15c989659880-manifests\") pod \"opendatahub-operator-controller-manager-67c7bcf47f-h6rbf\" (UID: \"c91ca7cb-43cc-439a-90e4-15c989659880\") " pod="opendatahub/opendatahub-operator-controller-manager-67c7bcf47f-h6rbf"
Sep 09 16:17:24.413136 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:24.412908    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8zfs\" (UniqueName: \"kubernetes.io/projected/c91ca7cb-43cc-439a-90e4-15c989659880-kube-api-access-b8zfs\") pod \"opendatahub-operator-controller-manager-67c7bcf47f-h6rbf\" (UID: \"c91ca7cb-43cc-439a-90e4-15c989659880\") " pod="opendatahub/opendatahub-operator-controller-manager-67c7bcf47f-h6rbf"
Sep 09 16:17:24.413136 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:24.412938    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c91ca7cb-43cc-439a-90e4-15c989659880-tmp\") pod \"opendatahub-operator-controller-manager-67c7bcf47f-h6rbf\" (UID: \"c91ca7cb-43cc-439a-90e4-15c989659880\") " pod="opendatahub/opendatahub-operator-controller-manager-67c7bcf47f-h6rbf"
Sep 09 16:17:24.413136 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:24.413034    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c91ca7cb-43cc-439a-90e4-15c989659880-apiservice-cert\") pod \"opendatahub-operator-controller-manager-67c7bcf47f-h6rbf\" (UID: \"c91ca7cb-43cc-439a-90e4-15c989659880\") " pod="opendatahub/opendatahub-operator-controller-manager-67c7bcf47f-h6rbf"
Sep 09 16:17:24.413136 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:24.413071    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c91ca7cb-43cc-439a-90e4-15c989659880-webhook-cert\") pod \"opendatahub-operator-controller-manager-67c7bcf47f-h6rbf\" (UID: \"c91ca7cb-43cc-439a-90e4-15c989659880\") " pod="opendatahub/opendatahub-operator-controller-manager-67c7bcf47f-h6rbf"
Sep 09 16:17:24.413369 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:24.413318    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c91ca7cb-43cc-439a-90e4-15c989659880-manifests\") pod \"opendatahub-operator-controller-manager-67c7bcf47f-h6rbf\" (UID: \"c91ca7cb-43cc-439a-90e4-15c989659880\") " pod="opendatahub/opendatahub-operator-controller-manager-67c7bcf47f-h6rbf"
Sep 09 16:17:24.415499 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:24.415448    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c91ca7cb-43cc-439a-90e4-15c989659880-tmp\") pod \"opendatahub-operator-controller-manager-67c7bcf47f-h6rbf\" (UID: \"c91ca7cb-43cc-439a-90e4-15c989659880\") " pod="opendatahub/opendatahub-operator-controller-manager-67c7bcf47f-h6rbf"
Sep 09 16:17:24.415702 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:24.415681    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c91ca7cb-43cc-439a-90e4-15c989659880-webhook-cert\") pod \"opendatahub-operator-controller-manager-67c7bcf47f-h6rbf\" (UID: \"c91ca7cb-43cc-439a-90e4-15c989659880\") " pod="opendatahub/opendatahub-operator-controller-manager-67c7bcf47f-h6rbf"
Sep 09 16:17:24.415749 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:24.415688    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c91ca7cb-43cc-439a-90e4-15c989659880-apiservice-cert\") pod \"opendatahub-operator-controller-manager-67c7bcf47f-h6rbf\" (UID: \"c91ca7cb-43cc-439a-90e4-15c989659880\") " pod="opendatahub/opendatahub-operator-controller-manager-67c7bcf47f-h6rbf"
Sep 09 16:17:24.435200 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:24.435169    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8zfs\" (UniqueName: \"kubernetes.io/projected/c91ca7cb-43cc-439a-90e4-15c989659880-kube-api-access-b8zfs\") pod \"opendatahub-operator-controller-manager-67c7bcf47f-h6rbf\" (UID: \"c91ca7cb-43cc-439a-90e4-15c989659880\") " pod="opendatahub/opendatahub-operator-controller-manager-67c7bcf47f-h6rbf"
Sep 09 16:17:24.477360 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:24.477324    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-67c7bcf47f-h6rbf"
Sep 09 16:17:24.784769 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:24.784691    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-67c7bcf47f-h6rbf"]
Sep 09 16:17:24.787964 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:17:24.787934    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc91ca7cb_43cc_439a_90e4_15c989659880.slice/crio-afdab02d8903cb334d1a50cfbfb858fabc559e9dc860bd6313d4a1bb5d78549d WatchSource:0}: Error finding container afdab02d8903cb334d1a50cfbfb858fabc559e9dc860bd6313d4a1bb5d78549d: Status 404 returned error can't find the container with id afdab02d8903cb334d1a50cfbfb858fabc559e9dc860bd6313d4a1bb5d78549d
Sep 09 16:17:24.829142 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:24.829108    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-67c7bcf47f-h6rbf" event={"ID":"c91ca7cb-43cc-439a-90e4-15c989659880","Type":"ContainerStarted","Data":"afdab02d8903cb334d1a50cfbfb858fabc559e9dc860bd6313d4a1bb5d78549d"}
Sep 09 16:17:27.843523 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:27.843495    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-67c7bcf47f-h6rbf" event={"ID":"c91ca7cb-43cc-439a-90e4-15c989659880","Type":"ContainerStarted","Data":"80422f8eb4c12682b4482a35568c7ea37ce9ab84660b9ac77a68930c10604eee"}
Sep 09 16:17:29.853208 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:29.853175    2567 generic.go:356] "Generic (PLEG): container finished" podID="c91ca7cb-43cc-439a-90e4-15c989659880" containerID="80422f8eb4c12682b4482a35568c7ea37ce9ab84660b9ac77a68930c10604eee" exitCode=0
Sep 09 16:17:29.853649 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:29.853254    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-67c7bcf47f-h6rbf" event={"ID":"c91ca7cb-43cc-439a-90e4-15c989659880","Type":"ContainerDied","Data":"80422f8eb4c12682b4482a35568c7ea37ce9ab84660b9ac77a68930c10604eee"}
Sep 09 16:17:30.859378 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:30.859347    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-67c7bcf47f-h6rbf" event={"ID":"c91ca7cb-43cc-439a-90e4-15c989659880","Type":"ContainerStarted","Data":"c21210686f80e4285bbea01929d413b1d4e5408700a64f113be175a94bf604f2"}
Sep 09 16:17:30.859762 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:30.859565    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-67c7bcf47f-h6rbf"
Sep 09 16:17:30.882731 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:30.882679    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-67c7bcf47f-h6rbf" podStartSLOduration=2.400518016 podStartE2EDuration="6.882661159s" totalImagesPullingTime="3.422240594s" totalInitContainerRuntime="1.059902549s" isStatefulPod=false podCreationTimestamp="2026-09-09 16:17:24 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 16:17:30.88128811 +0000 UTC m=+526.299138108" watchObservedRunningTime="2026-09-09 16:17:30.882661159 +0000 UTC m=+526.300511159"
Sep 09 16:17:31.817079 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:31.817051    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7bc87557cd-6bpm8"
Sep 09 16:17:41.866942 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:41.866866    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-67c7bcf47f-h6rbf"
Sep 09 16:17:52.696353 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:52.696316    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w"]
Sep 09 16:17:52.709698 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:52.709672    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w"
Sep 09 16:17:52.710441 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:52.710414    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w"]
Sep 09 16:17:52.712796 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:52.712773    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-wn25j\""
Sep 09 16:17:52.712918 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:52.712824    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 09 16:17:52.759792 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:52.759753    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/123f0ac2-1909-428d-a00e-3c9bff13a4f2-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w\" (UID: \"123f0ac2-1909-428d-a00e-3c9bff13a4f2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w"
Sep 09 16:17:52.759965 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:52.759804    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/123f0ac2-1909-428d-a00e-3c9bff13a4f2-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w\" (UID: \"123f0ac2-1909-428d-a00e-3c9bff13a4f2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w"
Sep 09 16:17:52.759965 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:52.759820    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/123f0ac2-1909-428d-a00e-3c9bff13a4f2-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w\" (UID: \"123f0ac2-1909-428d-a00e-3c9bff13a4f2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w"
Sep 09 16:17:52.759965 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:52.759892    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/123f0ac2-1909-428d-a00e-3c9bff13a4f2-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w\" (UID: \"123f0ac2-1909-428d-a00e-3c9bff13a4f2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w"
Sep 09 16:17:52.759965 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:52.759939    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/123f0ac2-1909-428d-a00e-3c9bff13a4f2-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w\" (UID: \"123f0ac2-1909-428d-a00e-3c9bff13a4f2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w"
Sep 09 16:17:52.760101 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:52.759978    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/123f0ac2-1909-428d-a00e-3c9bff13a4f2-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w\" (UID: \"123f0ac2-1909-428d-a00e-3c9bff13a4f2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w"
Sep 09 16:17:52.760101 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:52.760014    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/123f0ac2-1909-428d-a00e-3c9bff13a4f2-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w\" (UID: \"123f0ac2-1909-428d-a00e-3c9bff13a4f2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w"
Sep 09 16:17:52.760101 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:52.760046    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/123f0ac2-1909-428d-a00e-3c9bff13a4f2-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w\" (UID: \"123f0ac2-1909-428d-a00e-3c9bff13a4f2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w"
Sep 09 16:17:52.760101 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:52.760069    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjwgs\" (UniqueName: \"kubernetes.io/projected/123f0ac2-1909-428d-a00e-3c9bff13a4f2-kube-api-access-xjwgs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w\" (UID: \"123f0ac2-1909-428d-a00e-3c9bff13a4f2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w"
Sep 09 16:17:52.860722 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:52.860684    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjwgs\" (UniqueName: \"kubernetes.io/projected/123f0ac2-1909-428d-a00e-3c9bff13a4f2-kube-api-access-xjwgs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w\" (UID: \"123f0ac2-1909-428d-a00e-3c9bff13a4f2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w"
Sep 09 16:17:52.860900 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:52.860733    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/123f0ac2-1909-428d-a00e-3c9bff13a4f2-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w\" (UID: \"123f0ac2-1909-428d-a00e-3c9bff13a4f2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w"
Sep 09 16:17:52.860900 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:52.860785    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/123f0ac2-1909-428d-a00e-3c9bff13a4f2-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w\" (UID: \"123f0ac2-1909-428d-a00e-3c9bff13a4f2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w"
Sep 09 16:17:52.860900 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:52.860808    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/123f0ac2-1909-428d-a00e-3c9bff13a4f2-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w\" (UID: \"123f0ac2-1909-428d-a00e-3c9bff13a4f2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w"
Sep 09 16:17:52.860900 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:52.860836    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/123f0ac2-1909-428d-a00e-3c9bff13a4f2-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w\" (UID: \"123f0ac2-1909-428d-a00e-3c9bff13a4f2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w"
Sep 09 16:17:52.860900 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:52.860860    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/123f0ac2-1909-428d-a00e-3c9bff13a4f2-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w\" (UID: \"123f0ac2-1909-428d-a00e-3c9bff13a4f2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w"
Sep 09 16:17:52.861109 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:52.860908    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/123f0ac2-1909-428d-a00e-3c9bff13a4f2-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w\" (UID: \"123f0ac2-1909-428d-a00e-3c9bff13a4f2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w"
Sep 09 16:17:52.861109 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:52.860941    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/123f0ac2-1909-428d-a00e-3c9bff13a4f2-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w\" (UID: \"123f0ac2-1909-428d-a00e-3c9bff13a4f2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w"
Sep 09 16:17:52.861109 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:52.860983    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/123f0ac2-1909-428d-a00e-3c9bff13a4f2-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w\" (UID: \"123f0ac2-1909-428d-a00e-3c9bff13a4f2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w"
Sep 09 16:17:52.861344 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:52.861319    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/123f0ac2-1909-428d-a00e-3c9bff13a4f2-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w\" (UID: \"123f0ac2-1909-428d-a00e-3c9bff13a4f2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w"
Sep 09 16:17:52.861435 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:52.861340    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/123f0ac2-1909-428d-a00e-3c9bff13a4f2-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w\" (UID: \"123f0ac2-1909-428d-a00e-3c9bff13a4f2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w"
Sep 09 16:17:52.861435 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:52.861372    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/123f0ac2-1909-428d-a00e-3c9bff13a4f2-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w\" (UID: \"123f0ac2-1909-428d-a00e-3c9bff13a4f2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w"
Sep 09 16:17:52.861578 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:52.861448    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/123f0ac2-1909-428d-a00e-3c9bff13a4f2-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w\" (UID: \"123f0ac2-1909-428d-a00e-3c9bff13a4f2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w"
Sep 09 16:17:52.861793 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:52.861774    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/123f0ac2-1909-428d-a00e-3c9bff13a4f2-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w\" (UID: \"123f0ac2-1909-428d-a00e-3c9bff13a4f2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w"
Sep 09 16:17:52.863415 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:52.863388    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/123f0ac2-1909-428d-a00e-3c9bff13a4f2-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w\" (UID: \"123f0ac2-1909-428d-a00e-3c9bff13a4f2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w"
Sep 09 16:17:52.863578 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:52.863561    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/123f0ac2-1909-428d-a00e-3c9bff13a4f2-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w\" (UID: \"123f0ac2-1909-428d-a00e-3c9bff13a4f2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w"
Sep 09 16:17:52.870047 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:52.870021    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjwgs\" (UniqueName: \"kubernetes.io/projected/123f0ac2-1909-428d-a00e-3c9bff13a4f2-kube-api-access-xjwgs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w\" (UID: \"123f0ac2-1909-428d-a00e-3c9bff13a4f2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w"
Sep 09 16:17:52.870159 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:52.870114    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/123f0ac2-1909-428d-a00e-3c9bff13a4f2-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w\" (UID: \"123f0ac2-1909-428d-a00e-3c9bff13a4f2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w"
Sep 09 16:17:53.020815 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:53.020720    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w"
Sep 09 16:17:53.309334 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:53.309251    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w"]
Sep 09 16:17:53.313861 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:17:53.313828    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod123f0ac2_1909_428d_a00e_3c9bff13a4f2.slice/crio-d923bb78a33f88fa6c4edd77c69bb445d5710463eef3920527ee70cef300d5d6 WatchSource:0}: Error finding container d923bb78a33f88fa6c4edd77c69bb445d5710463eef3920527ee70cef300d5d6: Status 404 returned error can't find the container with id d923bb78a33f88fa6c4edd77c69bb445d5710463eef3920527ee70cef300d5d6
Sep 09 16:17:53.955682 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:53.955649    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w" event={"ID":"123f0ac2-1909-428d-a00e-3c9bff13a4f2","Type":"ContainerStarted","Data":"d923bb78a33f88fa6c4edd77c69bb445d5710463eef3920527ee70cef300d5d6"}
Sep 09 16:17:54.441909 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:54.441872    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-9b4w6"]
Sep 09 16:17:54.446680 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:54.446662    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-9b4w6"
Sep 09 16:17:54.450478 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:54.449761    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Sep 09 16:17:54.450478 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:54.449990    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-rb5wt\""
Sep 09 16:17:54.464074 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:54.464038    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-9b4w6"]
Sep 09 16:17:54.476385 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:54.476353    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f52x7\" (UniqueName: \"kubernetes.io/projected/d385db74-c01b-40d2-89f0-f71cca32b2c4-kube-api-access-f52x7\") pod \"odh-model-controller-7998c67dc4-9b4w6\" (UID: \"d385db74-c01b-40d2-89f0-f71cca32b2c4\") " pod="opendatahub/odh-model-controller-7998c67dc4-9b4w6"
Sep 09 16:17:54.476385 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:54.476389    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d385db74-c01b-40d2-89f0-f71cca32b2c4-cert\") pod \"odh-model-controller-7998c67dc4-9b4w6\" (UID: \"d385db74-c01b-40d2-89f0-f71cca32b2c4\") " pod="opendatahub/odh-model-controller-7998c67dc4-9b4w6"
Sep 09 16:17:54.577893 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:54.577853    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f52x7\" (UniqueName: \"kubernetes.io/projected/d385db74-c01b-40d2-89f0-f71cca32b2c4-kube-api-access-f52x7\") pod \"odh-model-controller-7998c67dc4-9b4w6\" (UID: \"d385db74-c01b-40d2-89f0-f71cca32b2c4\") " pod="opendatahub/odh-model-controller-7998c67dc4-9b4w6"
Sep 09 16:17:54.577893 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:54.577896    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d385db74-c01b-40d2-89f0-f71cca32b2c4-cert\") pod \"odh-model-controller-7998c67dc4-9b4w6\" (UID: \"d385db74-c01b-40d2-89f0-f71cca32b2c4\") " pod="opendatahub/odh-model-controller-7998c67dc4-9b4w6"
Sep 09 16:17:54.578103 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:17:54.578031    2567 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Sep 09 16:17:54.578103 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:17:54.578091    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d385db74-c01b-40d2-89f0-f71cca32b2c4-cert podName:d385db74-c01b-40d2-89f0-f71cca32b2c4 nodeName:}" failed. No retries permitted until 2026-09-09 16:17:55.078071243 +0000 UTC m=+550.495921221 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/d385db74-c01b-40d2-89f0-f71cca32b2c4-cert") pod "odh-model-controller-7998c67dc4-9b4w6" (UID: "d385db74-c01b-40d2-89f0-f71cca32b2c4") : secret "odh-model-controller-webhook-cert" not found
Sep 09 16:17:54.586088 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:54.586054    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f52x7\" (UniqueName: \"kubernetes.io/projected/d385db74-c01b-40d2-89f0-f71cca32b2c4-kube-api-access-f52x7\") pod \"odh-model-controller-7998c67dc4-9b4w6\" (UID: \"d385db74-c01b-40d2-89f0-f71cca32b2c4\") " pod="opendatahub/odh-model-controller-7998c67dc4-9b4w6"
Sep 09 16:17:54.995311 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:54.995277    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-6cmtl"]
Sep 09 16:17:54.998854 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:54.998834    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-6cmtl"
Sep 09 16:17:55.001228 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:55.001204    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Sep 09 16:17:55.010866 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:55.010842    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-6cmtl"]
Sep 09 16:17:55.082419 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:55.082381    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d385db74-c01b-40d2-89f0-f71cca32b2c4-cert\") pod \"odh-model-controller-7998c67dc4-9b4w6\" (UID: \"d385db74-c01b-40d2-89f0-f71cca32b2c4\") " pod="opendatahub/odh-model-controller-7998c67dc4-9b4w6"
Sep 09 16:17:55.082651 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:55.082505    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pw48b\" (UniqueName: \"kubernetes.io/projected/390a6dc0-1968-427b-b3b8-cc15b1201765-kube-api-access-pw48b\") pod \"model-serving-api-54d89bd5fc-6cmtl\" (UID: \"390a6dc0-1968-427b-b3b8-cc15b1201765\") " pod="opendatahub/model-serving-api-54d89bd5fc-6cmtl"
Sep 09 16:17:55.082651 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:55.082547    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/390a6dc0-1968-427b-b3b8-cc15b1201765-tls-certs\") pod \"model-serving-api-54d89bd5fc-6cmtl\" (UID: \"390a6dc0-1968-427b-b3b8-cc15b1201765\") " pod="opendatahub/model-serving-api-54d89bd5fc-6cmtl"
Sep 09 16:17:55.084993 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:55.084973    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d385db74-c01b-40d2-89f0-f71cca32b2c4-cert\") pod \"odh-model-controller-7998c67dc4-9b4w6\" (UID: \"d385db74-c01b-40d2-89f0-f71cca32b2c4\") " pod="opendatahub/odh-model-controller-7998c67dc4-9b4w6"
Sep 09 16:17:55.183363 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:55.183334    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pw48b\" (UniqueName: \"kubernetes.io/projected/390a6dc0-1968-427b-b3b8-cc15b1201765-kube-api-access-pw48b\") pod \"model-serving-api-54d89bd5fc-6cmtl\" (UID: \"390a6dc0-1968-427b-b3b8-cc15b1201765\") " pod="opendatahub/model-serving-api-54d89bd5fc-6cmtl"
Sep 09 16:17:55.183555 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:55.183375    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/390a6dc0-1968-427b-b3b8-cc15b1201765-tls-certs\") pod \"model-serving-api-54d89bd5fc-6cmtl\" (UID: \"390a6dc0-1968-427b-b3b8-cc15b1201765\") " pod="opendatahub/model-serving-api-54d89bd5fc-6cmtl"
Sep 09 16:17:55.183555 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:17:55.183540    2567 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Sep 09 16:17:55.183647 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:17:55.183620    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/390a6dc0-1968-427b-b3b8-cc15b1201765-tls-certs podName:390a6dc0-1968-427b-b3b8-cc15b1201765 nodeName:}" failed. No retries permitted until 2026-09-09 16:17:55.683599041 +0000 UTC m=+551.101449019 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/390a6dc0-1968-427b-b3b8-cc15b1201765-tls-certs") pod "model-serving-api-54d89bd5fc-6cmtl" (UID: "390a6dc0-1968-427b-b3b8-cc15b1201765") : secret "model-serving-api-tls" not found
Sep 09 16:17:55.193877 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:55.193852    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pw48b\" (UniqueName: \"kubernetes.io/projected/390a6dc0-1968-427b-b3b8-cc15b1201765-kube-api-access-pw48b\") pod \"model-serving-api-54d89bd5fc-6cmtl\" (UID: \"390a6dc0-1968-427b-b3b8-cc15b1201765\") " pod="opendatahub/model-serving-api-54d89bd5fc-6cmtl"
Sep 09 16:17:55.359716 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:55.359677    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-9b4w6"
Sep 09 16:17:55.636896 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:55.636872    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-9b4w6"]
Sep 09 16:17:55.639150 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:17:55.639120    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd385db74_c01b_40d2_89f0_f71cca32b2c4.slice/crio-775ebf2aedbb7b9ae9f4abdadcce34e3bcddac1f0f378ccc65cae2a3fab1add3 WatchSource:0}: Error finding container 775ebf2aedbb7b9ae9f4abdadcce34e3bcddac1f0f378ccc65cae2a3fab1add3: Status 404 returned error can't find the container with id 775ebf2aedbb7b9ae9f4abdadcce34e3bcddac1f0f378ccc65cae2a3fab1add3
Sep 09 16:17:55.685887 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:55.685860    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/390a6dc0-1968-427b-b3b8-cc15b1201765-tls-certs\") pod \"model-serving-api-54d89bd5fc-6cmtl\" (UID: \"390a6dc0-1968-427b-b3b8-cc15b1201765\") " pod="opendatahub/model-serving-api-54d89bd5fc-6cmtl"
Sep 09 16:17:55.688468 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:55.688429    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/390a6dc0-1968-427b-b3b8-cc15b1201765-tls-certs\") pod \"model-serving-api-54d89bd5fc-6cmtl\" (UID: \"390a6dc0-1968-427b-b3b8-cc15b1201765\") " pod="opendatahub/model-serving-api-54d89bd5fc-6cmtl"
Sep 09 16:17:55.909430 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:55.909331    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-6cmtl"
Sep 09 16:17:55.968648 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:55.968612    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-9b4w6" event={"ID":"d385db74-c01b-40d2-89f0-f71cca32b2c4","Type":"ContainerStarted","Data":"775ebf2aedbb7b9ae9f4abdadcce34e3bcddac1f0f378ccc65cae2a3fab1add3"}
Sep 09 16:17:56.184114 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:56.184026    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-6cmtl"]
Sep 09 16:17:56.385743 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:17:56.385697    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod390a6dc0_1968_427b_b3b8_cc15b1201765.slice/crio-6a848b007eb0d9a6350f65bb4df2576f60c858e2f05109fec68af47949fb3e1b WatchSource:0}: Error finding container 6a848b007eb0d9a6350f65bb4df2576f60c858e2f05109fec68af47949fb3e1b: Status 404 returned error can't find the container with id 6a848b007eb0d9a6350f65bb4df2576f60c858e2f05109fec68af47949fb3e1b
Sep 09 16:17:56.976613 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:17:56.976535    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-6cmtl" event={"ID":"390a6dc0-1968-427b-b3b8-cc15b1201765","Type":"ContainerStarted","Data":"6a848b007eb0d9a6350f65bb4df2576f60c858e2f05109fec68af47949fb3e1b"}
Sep 09 16:18:00.580643 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:18:00.580596    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:18:00.580984 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:18:00.580671    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:18:00.580984 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:18:00.580698    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:18:00.999727 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:18:00.999612    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-6cmtl" event={"ID":"390a6dc0-1968-427b-b3b8-cc15b1201765","Type":"ContainerStarted","Data":"69b02bb021d307d0b6f694c7ccac9bf913999db242af799a612c4c82173af963"}
Sep 09 16:18:00.999897 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:18:00.999731    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-6cmtl"
Sep 09 16:18:01.001599 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:18:01.001567    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w" event={"ID":"123f0ac2-1909-428d-a00e-3c9bff13a4f2","Type":"ContainerStarted","Data":"9b39b97b1193b64bdd616e3e9e65c618426db2221036e9a961475f044ee82739"}
Sep 09 16:18:01.003334 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:18:01.003310    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-9b4w6" event={"ID":"d385db74-c01b-40d2-89f0-f71cca32b2c4","Type":"ContainerStarted","Data":"1c0218f1865007ce279031d230a6245d851545b606bf3caea83bc29b4d41bdd8"}
Sep 09 16:18:01.003488 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:18:01.003373    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-9b4w6"
Sep 09 16:18:01.018893 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:18:01.018832    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-6cmtl" podStartSLOduration=2.784720104 podStartE2EDuration="7.018810967s" totalImagesPullingTime="4.234090863s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 16:17:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 16:18:01.018445928 +0000 UTC m=+556.436295942" watchObservedRunningTime="2026-09-09 16:18:01.018810967 +0000 UTC m=+556.436660969"
Sep 09 16:18:01.020932 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:18:01.020897    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w"
Sep 09 16:18:01.022394 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:18:01.022367    2567 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.22:15021/healthz/ready\": dial tcp 10.134.0.22:15021: connect: connection refused" start-of-body=
Sep 09 16:18:01.022523 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:18:01.022426    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w" podUID="123f0ac2-1909-428d-a00e-3c9bff13a4f2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.22:15021/healthz/ready\": dial tcp 10.134.0.22:15021: connect: connection refused"
Sep 09 16:18:01.059158 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:18:01.059105    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w" podStartSLOduration=1.794801382 podStartE2EDuration="9.059088224s" totalImagesPullingTime="7.264286842s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 16:17:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 16:18:01.058057457 +0000 UTC m=+556.475907466" watchObservedRunningTime="2026-09-09 16:18:01.059088224 +0000 UTC m=+556.476938242"
Sep 09 16:18:01.059320 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:18:01.059211    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-9b4w6" podStartSLOduration=2.113651268 podStartE2EDuration="7.059205832s" totalImagesPullingTime="4.945554564s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 16:17:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 16:18:01.037393654 +0000 UTC m=+556.455243653" watchObservedRunningTime="2026-09-09 16:18:01.059205832 +0000 UTC m=+556.477055868"
Sep 09 16:18:01.183607 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:18:01.183575    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-t9mz5"]
Sep 09 16:18:01.186996 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:18:01.186980    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-t9mz5"
Sep 09 16:18:01.189505 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:18:01.189475    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 09 16:18:01.189505 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:18:01.189502    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-s6w2w\""
Sep 09 16:18:01.189687 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:18:01.189486    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 09 16:18:01.189687 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:18:01.189477    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 09 16:18:01.201872 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:18:01.201843    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-t9mz5"]
Sep 09 16:18:01.237810 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:18:01.237774    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/aeddd9d7-76d4-41a8-baad-a13ce3180a50-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-t9mz5\" (UID: \"aeddd9d7-76d4-41a8-baad-a13ce3180a50\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-t9mz5"
Sep 09 16:18:01.237962 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:18:01.237829    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/aeddd9d7-76d4-41a8-baad-a13ce3180a50-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-t9mz5\" (UID: \"aeddd9d7-76d4-41a8-baad-a13ce3180a50\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-t9mz5"
Sep 09 16:18:01.237962 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:18:01.237888    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/aeddd9d7-76d4-41a8-baad-a13ce3180a50-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-t9mz5\" (UID: \"aeddd9d7-76d4-41a8-baad-a13ce3180a50\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-t9mz5"
Sep 09 16:18:01.237962 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:18:01.237921    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/aeddd9d7-76d4-41a8-baad-a13ce3180a50-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-t9mz5\" (UID: \"aeddd9d7-76d4-41a8-baad-a13ce3180a50\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-t9mz5"
Sep 09 16:18:01.238094 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:18:01.237965    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aeddd9d7-76d4-41a8-baad-a13ce3180a50-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-t9mz5\" (UID: \"aeddd9d7-76d4-41a8-baad-a13ce3180a50\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-t9mz5"
Sep 09 16:18:01.238094 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:18:01.237994    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/aeddd9d7-76d4-41a8-baad-a13ce3180a50-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-t9mz5\" (UID: \"aeddd9d7-76d4-41a8-baad-a13ce3180a50\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-t9mz5"
Sep 09 16:18:01.238094 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:18:01.238011    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgg69\" (UniqueName: \"kubernetes.io/projected/aeddd9d7-76d4-41a8-baad-a13ce3180a50-kube-api-access-kgg69\") pod \"istiod-openshift-gateway-5677ccb49b-t9mz5\" (UID: \"aeddd9d7-76d4-41a8-baad-a13ce3180a50\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-t9mz5"
Sep 09 16:18:01.339196 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:18:01.339166    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/aeddd9d7-76d4-41a8-baad-a13ce3180a50-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-t9mz5\" (UID: \"aeddd9d7-76d4-41a8-baad-a13ce3180a50\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-t9mz5"
Sep 09 16:18:01.339553 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:18:01.339204    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/aeddd9d7-76d4-41a8-baad-a13ce3180a50-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-t9mz5\" (UID: \"aeddd9d7-76d4-41a8-baad-a13ce3180a50\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-t9mz5"
Sep 09 16:18:01.339553 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:18:01.339234    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aeddd9d7-76d4-41a8-baad-a13ce3180a50-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-t9mz5\" (UID: \"aeddd9d7-76d4-41a8-baad-a13ce3180a50\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-t9mz5"
Sep 09 16:18:01.339553 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:18:01.339264    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/aeddd9d7-76d4-41a8-baad-a13ce3180a50-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-t9mz5\" (UID: \"aeddd9d7-76d4-41a8-baad-a13ce3180a50\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-t9mz5"
Sep 09 16:18:01.339553 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:18:01.339287    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgg69\" (UniqueName: \"kubernetes.io/projected/aeddd9d7-76d4-41a8-baad-a13ce3180a50-kube-api-access-kgg69\") pod \"istiod-openshift-gateway-5677ccb49b-t9mz5\" (UID: \"aeddd9d7-76d4-41a8-baad-a13ce3180a50\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-t9mz5"
Sep 09 16:18:01.339553 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:18:01.339324    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/aeddd9d7-76d4-41a8-baad-a13ce3180a50-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-t9mz5\" (UID: \"aeddd9d7-76d4-41a8-baad-a13ce3180a50\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-t9mz5"
Sep 09 16:18:01.339553 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:18:01.339377    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/aeddd9d7-76d4-41a8-baad-a13ce3180a50-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-t9mz5\" (UID: \"aeddd9d7-76d4-41a8-baad-a13ce3180a50\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-t9mz5"
Sep 09 16:18:01.340052 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:18:01.339990    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/aeddd9d7-76d4-41a8-baad-a13ce3180a50-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-t9mz5\" (UID: \"aeddd9d7-76d4-41a8-baad-a13ce3180a50\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-t9mz5"
Sep 09 16:18:01.341750 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:18:01.341728    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/aeddd9d7-76d4-41a8-baad-a13ce3180a50-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-t9mz5\" (UID: \"aeddd9d7-76d4-41a8-baad-a13ce3180a50\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-t9mz5"
Sep 09 16:18:01.341871 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:18:01.341854    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/aeddd9d7-76d4-41a8-baad-a13ce3180a50-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-t9mz5\" (UID: \"aeddd9d7-76d4-41a8-baad-a13ce3180a50\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-t9mz5"
Sep 09 16:18:01.342295 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:18:01.342275    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/aeddd9d7-76d4-41a8-baad-a13ce3180a50-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-t9mz5\" (UID: \"aeddd9d7-76d4-41a8-baad-a13ce3180a50\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-t9mz5"
Sep 09 16:18:01.342382 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:18:01.342337    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/aeddd9d7-76d4-41a8-baad-a13ce3180a50-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-t9mz5\" (UID: \"aeddd9d7-76d4-41a8-baad-a13ce3180a50\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-t9mz5"
Sep 09 16:18:01.347361 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:18:01.347338    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aeddd9d7-76d4-41a8-baad-a13ce3180a50-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-t9mz5\" (UID: \"aeddd9d7-76d4-41a8-baad-a13ce3180a50\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-t9mz5"
Sep 09 16:18:01.348154 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:18:01.348128    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgg69\" (UniqueName: \"kubernetes.io/projected/aeddd9d7-76d4-41a8-baad-a13ce3180a50-kube-api-access-kgg69\") pod \"istiod-openshift-gateway-5677ccb49b-t9mz5\" (UID: \"aeddd9d7-76d4-41a8-baad-a13ce3180a50\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-t9mz5"
Sep 09 16:18:01.497141 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:18:01.497107    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-t9mz5"
Sep 09 16:18:01.772566 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:18:01.772499    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-t9mz5"]
Sep 09 16:18:01.775020 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:18:01.774990    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaeddd9d7_76d4_41a8_baad_a13ce3180a50.slice/crio-9f3d083a12ef24c56ebf8ec32d0f4c861b577ca9d94966c4281268224f60a5cb WatchSource:0}: Error finding container 9f3d083a12ef24c56ebf8ec32d0f4c861b577ca9d94966c4281268224f60a5cb: Status 404 returned error can't find the container with id 9f3d083a12ef24c56ebf8ec32d0f4c861b577ca9d94966c4281268224f60a5cb
Sep 09 16:18:02.008925 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:18:02.008887    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-t9mz5" event={"ID":"aeddd9d7-76d4-41a8-baad-a13ce3180a50","Type":"ContainerStarted","Data":"9f3d083a12ef24c56ebf8ec32d0f4c861b577ca9d94966c4281268224f60a5cb"}
Sep 09 16:18:02.025692 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:18:02.025618    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w"
Sep 09 16:18:03.014410 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:18:03.014368    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w"
Sep 09 16:18:03.015497 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:18:03.015473    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w"
Sep 09 16:18:04.590739 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:18:04.590700    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:18:04.591033 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:18:04.590770    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:18:05.027085 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:18:05.026978    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-t9mz5" event={"ID":"aeddd9d7-76d4-41a8-baad-a13ce3180a50","Type":"ContainerStarted","Data":"14a74eb78c09dc59c0c70d8ecb2650255ffa26437e04727dfab68c1317b3ec93"}
Sep 09 16:18:05.027352 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:18:05.027313    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-t9mz5"
Sep 09 16:18:05.028866 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:18:05.028835    2567 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-t9mz5 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 09 16:18:05.028974 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:18:05.028897    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-t9mz5" podUID="aeddd9d7-76d4-41a8-baad-a13ce3180a50" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 16:18:05.071252 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:18:05.071197    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-t9mz5" podStartSLOduration=1.25791586 podStartE2EDuration="4.0711806s" totalImagesPullingTime="2.81326474s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 16:18:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 16:18:05.068152055 +0000 UTC m=+560.486002054" watchObservedRunningTime="2026-09-09 16:18:05.0711806 +0000 UTC m=+560.489030599"
Sep 09 16:18:06.033184 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:18:06.033151    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-t9mz5"
Sep 09 16:18:12.011347 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:18:12.011314    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-9b4w6"
Sep 09 16:18:12.014114 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:18:12.014095    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-6cmtl"
Sep 09 16:18:45.050077 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:18:45.049967    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-xwlzs_16ecfb4d-7944-465e-a38e-d8c5bda6b1ac/console-operator/2.log"
Sep 09 16:18:45.060247 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:18:45.051896    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-xwlzs_16ecfb4d-7944-465e-a38e-d8c5bda6b1ac/console-operator/2.log"
Sep 09 16:20:22.031225 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:20:22.031187    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:20:22.034840 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:20:22.034822    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"
Sep 09 16:20:22.037026 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:20:22.037001    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-bz25f\""
Sep 09 16:20:22.037026 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:20:22.037017    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 09 16:20:22.037026 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:20:22.037001    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 09 16:20:22.037784 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:20:22.037767    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 09 16:20:22.042782 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:20:22.042751    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:20:22.071163 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:20:22.071126    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:20:22.075743 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:20:22.075719    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e7f48af3-8fb3-4bb1-91f4-09478ea358f2-config-file\") pod \"limitador-limitador-84994f7ffc-n7dqd\" (UID: \"e7f48af3-8fb3-4bb1-91f4-09478ea358f2\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"
Sep 09 16:20:22.075810 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:20:22.075757    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4cmr\" (UniqueName: \"kubernetes.io/projected/e7f48af3-8fb3-4bb1-91f4-09478ea358f2-kube-api-access-g4cmr\") pod \"limitador-limitador-84994f7ffc-n7dqd\" (UID: \"e7f48af3-8fb3-4bb1-91f4-09478ea358f2\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"
Sep 09 16:20:22.176209 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:20:22.176181    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e7f48af3-8fb3-4bb1-91f4-09478ea358f2-config-file\") pod \"limitador-limitador-84994f7ffc-n7dqd\" (UID: \"e7f48af3-8fb3-4bb1-91f4-09478ea358f2\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"
Sep 09 16:20:22.176209 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:20:22.176212    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4cmr\" (UniqueName: \"kubernetes.io/projected/e7f48af3-8fb3-4bb1-91f4-09478ea358f2-kube-api-access-g4cmr\") pod \"limitador-limitador-84994f7ffc-n7dqd\" (UID: \"e7f48af3-8fb3-4bb1-91f4-09478ea358f2\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"
Sep 09 16:20:22.176844 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:20:22.176826    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e7f48af3-8fb3-4bb1-91f4-09478ea358f2-config-file\") pod \"limitador-limitador-84994f7ffc-n7dqd\" (UID: \"e7f48af3-8fb3-4bb1-91f4-09478ea358f2\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"
Sep 09 16:20:22.189440 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:20:22.189403    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4cmr\" (UniqueName: \"kubernetes.io/projected/e7f48af3-8fb3-4bb1-91f4-09478ea358f2-kube-api-access-g4cmr\") pod \"limitador-limitador-84994f7ffc-n7dqd\" (UID: \"e7f48af3-8fb3-4bb1-91f4-09478ea358f2\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"
Sep 09 16:20:22.345722 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:20:22.345689    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"
Sep 09 16:20:22.615474 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:20:22.615397    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:20:22.617742 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:20:22.617715    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7f48af3_8fb3_4bb1_91f4_09478ea358f2.slice/crio-aec0101878e725d1d6e01e80100135406b8cc35e79cad4b788691b11f4eb88b6 WatchSource:0}: Error finding container aec0101878e725d1d6e01e80100135406b8cc35e79cad4b788691b11f4eb88b6: Status 404 returned error can't find the container with id aec0101878e725d1d6e01e80100135406b8cc35e79cad4b788691b11f4eb88b6
Sep 09 16:20:22.619621 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:20:22.619603    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 16:20:22.629143 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:20:22.629097    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-n7dqd" event={"ID":"e7f48af3-8fb3-4bb1-91f4-09478ea358f2","Type":"ContainerStarted","Data":"aec0101878e725d1d6e01e80100135406b8cc35e79cad4b788691b11f4eb88b6"}
Sep 09 16:20:25.647304 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:20:25.647266    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-n7dqd" event={"ID":"e7f48af3-8fb3-4bb1-91f4-09478ea358f2","Type":"ContainerStarted","Data":"4319a808421eaea27a2df343346a7387d4b30ddd2fc50065af75555c5251cab1"}
Sep 09 16:20:25.647710 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:20:25.647342    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"
Sep 09 16:20:25.663159 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:20:25.663113    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-84994f7ffc-n7dqd" podStartSLOduration=1.377755614 podStartE2EDuration="3.663074206s" totalImagesPullingTime="2.285318592s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 16:20:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 16:20:25.662131887 +0000 UTC m=+701.079981884" watchObservedRunningTime="2026-09-09 16:20:25.663074206 +0000 UTC m=+701.080924206"
Sep 09 16:20:36.652503 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:20:36.652400    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"
Sep 09 16:21:44.390873 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:21:44.390830    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:22:09.435209 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:22:09.435126    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:22:28.203635 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:22:28.203591    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-vhgp4"]
Sep 09 16:22:28.207383 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:22:28.207361    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vhgp4"
Sep 09 16:22:28.210247 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:22:28.210213    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-m45r2\""
Sep 09 16:22:28.210386 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:22:28.210213    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 09 16:22:28.210386 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:22:28.210216    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 09 16:22:28.210935 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:22:28.210922    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 09 16:22:28.218521 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:22:28.218490    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-vhgp4"]
Sep 09 16:22:28.315103 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:22:28.315051    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5394245-74ab-4f80-9e3a-9b16493e13e5-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vhgp4\" (UID: \"a5394245-74ab-4f80-9e3a-9b16493e13e5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vhgp4"
Sep 09 16:22:28.315103 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:22:28.315097    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2p5bc\" (UniqueName: \"kubernetes.io/projected/a5394245-74ab-4f80-9e3a-9b16493e13e5-kube-api-access-2p5bc\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vhgp4\" (UID: \"a5394245-74ab-4f80-9e3a-9b16493e13e5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vhgp4"
Sep 09 16:22:28.315364 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:22:28.315238    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5394245-74ab-4f80-9e3a-9b16493e13e5-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vhgp4\" (UID: \"a5394245-74ab-4f80-9e3a-9b16493e13e5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vhgp4"
Sep 09 16:22:28.315364 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:22:28.315294    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5394245-74ab-4f80-9e3a-9b16493e13e5-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vhgp4\" (UID: \"a5394245-74ab-4f80-9e3a-9b16493e13e5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vhgp4"
Sep 09 16:22:28.315364 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:22:28.315356    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5394245-74ab-4f80-9e3a-9b16493e13e5-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vhgp4\" (UID: \"a5394245-74ab-4f80-9e3a-9b16493e13e5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vhgp4"
Sep 09 16:22:28.315557 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:22:28.315434    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5394245-74ab-4f80-9e3a-9b16493e13e5-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vhgp4\" (UID: \"a5394245-74ab-4f80-9e3a-9b16493e13e5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vhgp4"
Sep 09 16:22:28.416667 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:22:28.416625    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5394245-74ab-4f80-9e3a-9b16493e13e5-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vhgp4\" (UID: \"a5394245-74ab-4f80-9e3a-9b16493e13e5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vhgp4"
Sep 09 16:22:28.416667 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:22:28.416671    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5394245-74ab-4f80-9e3a-9b16493e13e5-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vhgp4\" (UID: \"a5394245-74ab-4f80-9e3a-9b16493e13e5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vhgp4"
Sep 09 16:22:28.416897 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:22:28.416710    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5394245-74ab-4f80-9e3a-9b16493e13e5-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vhgp4\" (UID: \"a5394245-74ab-4f80-9e3a-9b16493e13e5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vhgp4"
Sep 09 16:22:28.416897 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:22:28.416730    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5394245-74ab-4f80-9e3a-9b16493e13e5-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vhgp4\" (UID: \"a5394245-74ab-4f80-9e3a-9b16493e13e5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vhgp4"
Sep 09 16:22:28.416897 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:22:28.416764    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5394245-74ab-4f80-9e3a-9b16493e13e5-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vhgp4\" (UID: \"a5394245-74ab-4f80-9e3a-9b16493e13e5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vhgp4"
Sep 09 16:22:28.416897 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:22:28.416787    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2p5bc\" (UniqueName: \"kubernetes.io/projected/a5394245-74ab-4f80-9e3a-9b16493e13e5-kube-api-access-2p5bc\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vhgp4\" (UID: \"a5394245-74ab-4f80-9e3a-9b16493e13e5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vhgp4"
Sep 09 16:22:28.417182 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:22:28.417161    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5394245-74ab-4f80-9e3a-9b16493e13e5-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vhgp4\" (UID: \"a5394245-74ab-4f80-9e3a-9b16493e13e5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vhgp4"
Sep 09 16:22:28.417246 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:22:28.417191    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5394245-74ab-4f80-9e3a-9b16493e13e5-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vhgp4\" (UID: \"a5394245-74ab-4f80-9e3a-9b16493e13e5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vhgp4"
Sep 09 16:22:28.417299 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:22:28.417258    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5394245-74ab-4f80-9e3a-9b16493e13e5-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vhgp4\" (UID: \"a5394245-74ab-4f80-9e3a-9b16493e13e5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vhgp4"
Sep 09 16:22:28.419161 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:22:28.419136    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5394245-74ab-4f80-9e3a-9b16493e13e5-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vhgp4\" (UID: \"a5394245-74ab-4f80-9e3a-9b16493e13e5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vhgp4"
Sep 09 16:22:28.419412 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:22:28.419396    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5394245-74ab-4f80-9e3a-9b16493e13e5-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vhgp4\" (UID: \"a5394245-74ab-4f80-9e3a-9b16493e13e5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vhgp4"
Sep 09 16:22:28.425223 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:22:28.425201    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2p5bc\" (UniqueName: \"kubernetes.io/projected/a5394245-74ab-4f80-9e3a-9b16493e13e5-kube-api-access-2p5bc\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vhgp4\" (UID: \"a5394245-74ab-4f80-9e3a-9b16493e13e5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vhgp4"
Sep 09 16:22:28.518579 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:22:28.518479    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vhgp4"
Sep 09 16:22:28.790064 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:22:28.790040    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-vhgp4"]
Sep 09 16:22:28.792313 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:22:28.792280    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5394245_74ab_4f80_9e3a_9b16493e13e5.slice/crio-e488a51df2db0343793017c5f0be756d3b9b4b96f04c09669a33d80bb850d996 WatchSource:0}: Error finding container e488a51df2db0343793017c5f0be756d3b9b4b96f04c09669a33d80bb850d996: Status 404 returned error can't find the container with id e488a51df2db0343793017c5f0be756d3b9b4b96f04c09669a33d80bb850d996
Sep 09 16:22:28.904449 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:22:28.904410    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:22:29.187519 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:22:29.187481    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vhgp4" event={"ID":"a5394245-74ab-4f80-9e3a-9b16493e13e5","Type":"ContainerStarted","Data":"e488a51df2db0343793017c5f0be756d3b9b4b96f04c09669a33d80bb850d996"}
Sep 09 16:22:30.193978 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:22:30.193889    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vhgp4" event={"ID":"a5394245-74ab-4f80-9e3a-9b16493e13e5","Type":"ContainerStarted","Data":"cc6b552206ea30674e0a3ab852a8755784b7051e804e188f1757b244041c9d29"}
Sep 09 16:22:30.209925 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:22:30.209877    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vhgp4" podStartSLOduration=1.179292365 podStartE2EDuration="2.209862054s" totalImagesPullingTime="1.030569689s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 16:22:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 16:22:30.209259155 +0000 UTC m=+825.627109155" watchObservedRunningTime="2026-09-09 16:22:30.209862054 +0000 UTC m=+825.627712052"
Sep 09 16:22:33.517320 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:22:33.517284    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:22:38.519606 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:22:38.519559    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vhgp4"
Sep 09 16:22:38.520026 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:22:38.519908    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vhgp4"
Sep 09 16:22:38.532604 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:22:38.532578    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vhgp4"
Sep 09 16:22:39.245802 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:22:39.245773    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vhgp4"
Sep 09 16:22:43.213715 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:22:43.213680    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:22:55.915027 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:22:55.914993    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:23:03.705877 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:23:03.705835    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-979n2"]
Sep 09 16:23:03.710144 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:23:03.710120    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-979n2"
Sep 09 16:23:03.712715 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:23:03.712690    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Sep 09 16:23:03.718227 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:23:03.718206    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-979n2"]
Sep 09 16:23:03.843538 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:23:03.843489    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgbzh\" (UniqueName: \"kubernetes.io/projected/8e7d0d70-2fbc-4f2b-8e12-45103aaf9446-kube-api-access-cgbzh\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-979n2\" (UID: \"8e7d0d70-2fbc-4f2b-8e12-45103aaf9446\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-979n2"
Sep 09 16:23:03.843538 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:23:03.843541    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8e7d0d70-2fbc-4f2b-8e12-45103aaf9446-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-979n2\" (UID: \"8e7d0d70-2fbc-4f2b-8e12-45103aaf9446\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-979n2"
Sep 09 16:23:03.843752 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:23:03.843639    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8e7d0d70-2fbc-4f2b-8e12-45103aaf9446-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-979n2\" (UID: \"8e7d0d70-2fbc-4f2b-8e12-45103aaf9446\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-979n2"
Sep 09 16:23:03.843752 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:23:03.843674    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e7d0d70-2fbc-4f2b-8e12-45103aaf9446-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-979n2\" (UID: \"8e7d0d70-2fbc-4f2b-8e12-45103aaf9446\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-979n2"
Sep 09 16:23:03.843752 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:23:03.843735    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e7d0d70-2fbc-4f2b-8e12-45103aaf9446-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-979n2\" (UID: \"8e7d0d70-2fbc-4f2b-8e12-45103aaf9446\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-979n2"
Sep 09 16:23:03.843851 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:23:03.843781    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8e7d0d70-2fbc-4f2b-8e12-45103aaf9446-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-979n2\" (UID: \"8e7d0d70-2fbc-4f2b-8e12-45103aaf9446\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-979n2"
Sep 09 16:23:03.945135 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:23:03.945091    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8e7d0d70-2fbc-4f2b-8e12-45103aaf9446-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-979n2\" (UID: \"8e7d0d70-2fbc-4f2b-8e12-45103aaf9446\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-979n2"
Sep 09 16:23:03.945331 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:23:03.945153    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8e7d0d70-2fbc-4f2b-8e12-45103aaf9446-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-979n2\" (UID: \"8e7d0d70-2fbc-4f2b-8e12-45103aaf9446\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-979n2"
Sep 09 16:23:03.945331 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:23:03.945173    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e7d0d70-2fbc-4f2b-8e12-45103aaf9446-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-979n2\" (UID: \"8e7d0d70-2fbc-4f2b-8e12-45103aaf9446\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-979n2"
Sep 09 16:23:03.945331 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:23:03.945206    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e7d0d70-2fbc-4f2b-8e12-45103aaf9446-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-979n2\" (UID: \"8e7d0d70-2fbc-4f2b-8e12-45103aaf9446\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-979n2"
Sep 09 16:23:03.945331 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:23:03.945312    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8e7d0d70-2fbc-4f2b-8e12-45103aaf9446-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-979n2\" (UID: \"8e7d0d70-2fbc-4f2b-8e12-45103aaf9446\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-979n2"
Sep 09 16:23:03.945591 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:23:03.945377    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgbzh\" (UniqueName: \"kubernetes.io/projected/8e7d0d70-2fbc-4f2b-8e12-45103aaf9446-kube-api-access-cgbzh\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-979n2\" (UID: \"8e7d0d70-2fbc-4f2b-8e12-45103aaf9446\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-979n2"
Sep 09 16:23:03.945647 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:23:03.945606    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8e7d0d70-2fbc-4f2b-8e12-45103aaf9446-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-979n2\" (UID: \"8e7d0d70-2fbc-4f2b-8e12-45103aaf9446\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-979n2"
Sep 09 16:23:03.945647 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:23:03.945618    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e7d0d70-2fbc-4f2b-8e12-45103aaf9446-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-979n2\" (UID: \"8e7d0d70-2fbc-4f2b-8e12-45103aaf9446\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-979n2"
Sep 09 16:23:03.945738 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:23:03.945683    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8e7d0d70-2fbc-4f2b-8e12-45103aaf9446-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-979n2\" (UID: \"8e7d0d70-2fbc-4f2b-8e12-45103aaf9446\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-979n2"
Sep 09 16:23:03.947779 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:23:03.947754    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8e7d0d70-2fbc-4f2b-8e12-45103aaf9446-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-979n2\" (UID: \"8e7d0d70-2fbc-4f2b-8e12-45103aaf9446\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-979n2"
Sep 09 16:23:03.947960 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:23:03.947943    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e7d0d70-2fbc-4f2b-8e12-45103aaf9446-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-979n2\" (UID: \"8e7d0d70-2fbc-4f2b-8e12-45103aaf9446\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-979n2"
Sep 09 16:23:03.953806 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:23:03.953783    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgbzh\" (UniqueName: \"kubernetes.io/projected/8e7d0d70-2fbc-4f2b-8e12-45103aaf9446-kube-api-access-cgbzh\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-979n2\" (UID: \"8e7d0d70-2fbc-4f2b-8e12-45103aaf9446\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-979n2"
Sep 09 16:23:04.021967 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:23:04.021855    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-979n2"
Sep 09 16:23:04.294469 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:23:04.294433    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-979n2"]
Sep 09 16:23:04.296676 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:23:04.296650    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e7d0d70_2fbc_4f2b_8e12_45103aaf9446.slice/crio-1c0fa2f7fe07592d55eb1bc37ead9140a9f028222e0080fe7510038f9cd80f70 WatchSource:0}: Error finding container 1c0fa2f7fe07592d55eb1bc37ead9140a9f028222e0080fe7510038f9cd80f70: Status 404 returned error can't find the container with id 1c0fa2f7fe07592d55eb1bc37ead9140a9f028222e0080fe7510038f9cd80f70
Sep 09 16:23:04.343532 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:23:04.343501    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-979n2" event={"ID":"8e7d0d70-2fbc-4f2b-8e12-45103aaf9446","Type":"ContainerStarted","Data":"1c0fa2f7fe07592d55eb1bc37ead9140a9f028222e0080fe7510038f9cd80f70"}
Sep 09 16:23:05.349526 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:23:05.349487    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-979n2" event={"ID":"8e7d0d70-2fbc-4f2b-8e12-45103aaf9446","Type":"ContainerStarted","Data":"4f91c9666fc936d868e5184cf332447a8b224dd7b5cd3cfff84502406429cd69"}
Sep 09 16:23:05.375165 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:23:05.375105    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-979n2" podStartSLOduration=2.134272621 podStartE2EDuration="2.375061685s" totalImagesPullingTime="240.789064ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 16:23:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 16:23:05.37266041 +0000 UTC m=+860.790510421" watchObservedRunningTime="2026-09-09 16:23:05.375061685 +0000 UTC m=+860.792911684"
Sep 09 16:23:05.531406 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:23:05.531369    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:23:12.422156 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:23:12.422118    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:23:14.022374 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:23:14.022329    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-979n2"
Sep 09 16:23:14.022844 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:23:14.022401    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-979n2"
Sep 09 16:23:14.035271 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:23:14.035234    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-979n2"
Sep 09 16:23:14.402818 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:23:14.402789    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-979n2"
Sep 09 16:23:45.077216 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:23:45.077191    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-xwlzs_16ecfb4d-7944-465e-a38e-d8c5bda6b1ac/console-operator/2.log"
Sep 09 16:23:45.083206 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:23:45.083177    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-xwlzs_16ecfb4d-7944-465e-a38e-d8c5bda6b1ac/console-operator/2.log"
Sep 09 16:24:56.529533 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:24:56.529497    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl"]
Sep 09 16:24:56.532943 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:24:56.532918    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl"
Sep 09 16:24:56.535264 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:24:56.535243    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-6d44b6-openshift-default-dockercfg-swpdl\""
Sep 09 16:24:56.545360 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:24:56.545336    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl"]
Sep 09 16:24:56.643977 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:24:56.643943    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bq578\" (UniqueName: \"kubernetes.io/projected/f1e36da0-3a7c-4765-aeb6-0b6339a09cc4-kube-api-access-bq578\") pod \"e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl\" (UID: \"f1e36da0-3a7c-4765-aeb6-0b6339a09cc4\") " pod="openshift-ingress/e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl"
Sep 09 16:24:56.644157 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:24:56.644060    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1e36da0-3a7c-4765-aeb6-0b6339a09cc4-credential-socket\") pod \"e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl\" (UID: \"f1e36da0-3a7c-4765-aeb6-0b6339a09cc4\") " pod="openshift-ingress/e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl"
Sep 09 16:24:56.644157 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:24:56.644137    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1e36da0-3a7c-4765-aeb6-0b6339a09cc4-workload-socket\") pod \"e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl\" (UID: \"f1e36da0-3a7c-4765-aeb6-0b6339a09cc4\") " pod="openshift-ingress/e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl"
Sep 09 16:24:56.644250 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:24:56.644188    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1e36da0-3a7c-4765-aeb6-0b6339a09cc4-istio-envoy\") pod \"e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl\" (UID: \"f1e36da0-3a7c-4765-aeb6-0b6339a09cc4\") " pod="openshift-ingress/e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl"
Sep 09 16:24:56.644250 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:24:56.644230    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1e36da0-3a7c-4765-aeb6-0b6339a09cc4-istio-podinfo\") pod \"e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl\" (UID: \"f1e36da0-3a7c-4765-aeb6-0b6339a09cc4\") " pod="openshift-ingress/e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl"
Sep 09 16:24:56.644318 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:24:56.644254    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1e36da0-3a7c-4765-aeb6-0b6339a09cc4-istio-token\") pod \"e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl\" (UID: \"f1e36da0-3a7c-4765-aeb6-0b6339a09cc4\") " pod="openshift-ingress/e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl"
Sep 09 16:24:56.644318 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:24:56.644294    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1e36da0-3a7c-4765-aeb6-0b6339a09cc4-workload-certs\") pod \"e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl\" (UID: \"f1e36da0-3a7c-4765-aeb6-0b6339a09cc4\") " pod="openshift-ingress/e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl"
Sep 09 16:24:56.644318 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:24:56.644312    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1e36da0-3a7c-4765-aeb6-0b6339a09cc4-istio-data\") pod \"e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl\" (UID: \"f1e36da0-3a7c-4765-aeb6-0b6339a09cc4\") " pod="openshift-ingress/e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl"
Sep 09 16:24:56.644410 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:24:56.644328    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1e36da0-3a7c-4765-aeb6-0b6339a09cc4-istiod-ca-cert\") pod \"e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl\" (UID: \"f1e36da0-3a7c-4765-aeb6-0b6339a09cc4\") " pod="openshift-ingress/e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl"
Sep 09 16:24:56.745105 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:24:56.745068    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1e36da0-3a7c-4765-aeb6-0b6339a09cc4-credential-socket\") pod \"e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl\" (UID: \"f1e36da0-3a7c-4765-aeb6-0b6339a09cc4\") " pod="openshift-ingress/e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl"
Sep 09 16:24:56.745303 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:24:56.745125    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1e36da0-3a7c-4765-aeb6-0b6339a09cc4-workload-socket\") pod \"e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl\" (UID: \"f1e36da0-3a7c-4765-aeb6-0b6339a09cc4\") " pod="openshift-ingress/e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl"
Sep 09 16:24:56.745303 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:24:56.745163    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1e36da0-3a7c-4765-aeb6-0b6339a09cc4-istio-envoy\") pod \"e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl\" (UID: \"f1e36da0-3a7c-4765-aeb6-0b6339a09cc4\") " pod="openshift-ingress/e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl"
Sep 09 16:24:56.745303 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:24:56.745188    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1e36da0-3a7c-4765-aeb6-0b6339a09cc4-istio-podinfo\") pod \"e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl\" (UID: \"f1e36da0-3a7c-4765-aeb6-0b6339a09cc4\") " pod="openshift-ingress/e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl"
Sep 09 16:24:56.745303 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:24:56.745215    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1e36da0-3a7c-4765-aeb6-0b6339a09cc4-istio-token\") pod \"e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl\" (UID: \"f1e36da0-3a7c-4765-aeb6-0b6339a09cc4\") " pod="openshift-ingress/e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl"
Sep 09 16:24:56.745303 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:24:56.745249    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1e36da0-3a7c-4765-aeb6-0b6339a09cc4-workload-certs\") pod \"e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl\" (UID: \"f1e36da0-3a7c-4765-aeb6-0b6339a09cc4\") " pod="openshift-ingress/e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl"
Sep 09 16:24:56.745303 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:24:56.745271    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1e36da0-3a7c-4765-aeb6-0b6339a09cc4-istio-data\") pod \"e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl\" (UID: \"f1e36da0-3a7c-4765-aeb6-0b6339a09cc4\") " pod="openshift-ingress/e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl"
Sep 09 16:24:56.745303 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:24:56.745296    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1e36da0-3a7c-4765-aeb6-0b6339a09cc4-istiod-ca-cert\") pod \"e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl\" (UID: \"f1e36da0-3a7c-4765-aeb6-0b6339a09cc4\") " pod="openshift-ingress/e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl"
Sep 09 16:24:56.745704 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:24:56.745366    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bq578\" (UniqueName: \"kubernetes.io/projected/f1e36da0-3a7c-4765-aeb6-0b6339a09cc4-kube-api-access-bq578\") pod \"e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl\" (UID: \"f1e36da0-3a7c-4765-aeb6-0b6339a09cc4\") " pod="openshift-ingress/e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl"
Sep 09 16:24:56.745704 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:24:56.745570    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1e36da0-3a7c-4765-aeb6-0b6339a09cc4-credential-socket\") pod \"e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl\" (UID: \"f1e36da0-3a7c-4765-aeb6-0b6339a09cc4\") " pod="openshift-ingress/e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl"
Sep 09 16:24:56.745704 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:24:56.745639    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1e36da0-3a7c-4765-aeb6-0b6339a09cc4-workload-socket\") pod \"e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl\" (UID: \"f1e36da0-3a7c-4765-aeb6-0b6339a09cc4\") " pod="openshift-ingress/e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl"
Sep 09 16:24:56.745863 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:24:56.745747    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1e36da0-3a7c-4765-aeb6-0b6339a09cc4-workload-certs\") pod \"e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl\" (UID: \"f1e36da0-3a7c-4765-aeb6-0b6339a09cc4\") " pod="openshift-ingress/e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl"
Sep 09 16:24:56.745863 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:24:56.745793    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1e36da0-3a7c-4765-aeb6-0b6339a09cc4-istio-data\") pod \"e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl\" (UID: \"f1e36da0-3a7c-4765-aeb6-0b6339a09cc4\") " pod="openshift-ingress/e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl"
Sep 09 16:24:56.746124 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:24:56.746075    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1e36da0-3a7c-4765-aeb6-0b6339a09cc4-istiod-ca-cert\") pod \"e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl\" (UID: \"f1e36da0-3a7c-4765-aeb6-0b6339a09cc4\") " pod="openshift-ingress/e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl"
Sep 09 16:24:56.747887 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:24:56.747863    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1e36da0-3a7c-4765-aeb6-0b6339a09cc4-istio-podinfo\") pod \"e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl\" (UID: \"f1e36da0-3a7c-4765-aeb6-0b6339a09cc4\") " pod="openshift-ingress/e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl"
Sep 09 16:24:56.748071 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:24:56.748055    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1e36da0-3a7c-4765-aeb6-0b6339a09cc4-istio-envoy\") pod \"e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl\" (UID: \"f1e36da0-3a7c-4765-aeb6-0b6339a09cc4\") " pod="openshift-ingress/e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl"
Sep 09 16:24:56.753231 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:24:56.753204    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1e36da0-3a7c-4765-aeb6-0b6339a09cc4-istio-token\") pod \"e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl\" (UID: \"f1e36da0-3a7c-4765-aeb6-0b6339a09cc4\") " pod="openshift-ingress/e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl"
Sep 09 16:24:56.753383 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:24:56.753363    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bq578\" (UniqueName: \"kubernetes.io/projected/f1e36da0-3a7c-4765-aeb6-0b6339a09cc4-kube-api-access-bq578\") pod \"e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl\" (UID: \"f1e36da0-3a7c-4765-aeb6-0b6339a09cc4\") " pod="openshift-ingress/e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl"
Sep 09 16:24:56.844449 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:24:56.844407    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl"
Sep 09 16:24:57.120868 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:24:57.120842    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl"]
Sep 09 16:24:57.123405 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:24:57.123380    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1e36da0_3a7c_4765_aeb6_0b6339a09cc4.slice/crio-9b28b2b656187d62207b5afa3d39cb901eceb336cb9e2ca49b526fb944a02c06 WatchSource:0}: Error finding container 9b28b2b656187d62207b5afa3d39cb901eceb336cb9e2ca49b526fb944a02c06: Status 404 returned error can't find the container with id 9b28b2b656187d62207b5afa3d39cb901eceb336cb9e2ca49b526fb944a02c06
Sep 09 16:24:57.125778 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:24:57.125735    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:24:57.125850 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:24:57.125824    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:24:57.125892 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:24:57.125865    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:24:57.850173 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:24:57.850141    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl" event={"ID":"f1e36da0-3a7c-4765-aeb6-0b6339a09cc4","Type":"ContainerStarted","Data":"6ddf2705b7bd4446dc2110af85a6252fde2d8f77b3c9729524b6a6a6c72f880b"}
Sep 09 16:24:57.850575 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:24:57.850193    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl" event={"ID":"f1e36da0-3a7c-4765-aeb6-0b6339a09cc4","Type":"ContainerStarted","Data":"9b28b2b656187d62207b5afa3d39cb901eceb336cb9e2ca49b526fb944a02c06"}
Sep 09 16:24:57.871387 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:24:57.871331    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl" podStartSLOduration=1.871313437 podStartE2EDuration="1.871313437s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 16:24:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 16:24:57.86928014 +0000 UTC m=+973.287130139" watchObservedRunningTime="2026-09-09 16:24:57.871313437 +0000 UTC m=+973.289163436"
Sep 09 16:24:58.845006 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:24:58.844970    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl"
Sep 09 16:24:58.850025 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:24:58.849999    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl"
Sep 09 16:24:58.854446 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:24:58.854422    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl"
Sep 09 16:24:58.855270 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:24:58.855254    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl"
Sep 09 16:25:00.122427 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:00.122390    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:25:04.755104 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:04.755073    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8"]
Sep 09 16:25:04.758811 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:04.758785    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8"
Sep 09 16:25:04.761767 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:04.761743    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-0d2e38e1-openshift-default-dockercfg-6bd8f\""
Sep 09 16:25:04.770321 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:04.770288    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8"]
Sep 09 16:25:04.823802 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:04.823756    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf6e0304-8dce-4764-8942-133cded65ce5-workload-socket\") pod \"e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8\" (UID: \"cf6e0304-8dce-4764-8942-133cded65ce5\") " pod="openshift-ingress/e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8"
Sep 09 16:25:04.823983 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:04.823814    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf6e0304-8dce-4764-8942-133cded65ce5-credential-socket\") pod \"e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8\" (UID: \"cf6e0304-8dce-4764-8942-133cded65ce5\") " pod="openshift-ingress/e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8"
Sep 09 16:25:04.823983 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:04.823841    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf6e0304-8dce-4764-8942-133cded65ce5-istio-podinfo\") pod \"e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8\" (UID: \"cf6e0304-8dce-4764-8942-133cded65ce5\") " pod="openshift-ingress/e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8"
Sep 09 16:25:04.823983 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:04.823916    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf6e0304-8dce-4764-8942-133cded65ce5-istiod-ca-cert\") pod \"e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8\" (UID: \"cf6e0304-8dce-4764-8942-133cded65ce5\") " pod="openshift-ingress/e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8"
Sep 09 16:25:04.823983 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:04.823976    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbqrs\" (UniqueName: \"kubernetes.io/projected/cf6e0304-8dce-4764-8942-133cded65ce5-kube-api-access-cbqrs\") pod \"e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8\" (UID: \"cf6e0304-8dce-4764-8942-133cded65ce5\") " pod="openshift-ingress/e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8"
Sep 09 16:25:04.824203 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:04.824033    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf6e0304-8dce-4764-8942-133cded65ce5-istio-envoy\") pod \"e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8\" (UID: \"cf6e0304-8dce-4764-8942-133cded65ce5\") " pod="openshift-ingress/e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8"
Sep 09 16:25:04.824203 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:04.824113    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf6e0304-8dce-4764-8942-133cded65ce5-istio-token\") pod \"e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8\" (UID: \"cf6e0304-8dce-4764-8942-133cded65ce5\") " pod="openshift-ingress/e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8"
Sep 09 16:25:04.824203 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:04.824169    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf6e0304-8dce-4764-8942-133cded65ce5-workload-certs\") pod \"e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8\" (UID: \"cf6e0304-8dce-4764-8942-133cded65ce5\") " pod="openshift-ingress/e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8"
Sep 09 16:25:04.824337 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:04.824204    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf6e0304-8dce-4764-8942-133cded65ce5-istio-data\") pod \"e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8\" (UID: \"cf6e0304-8dce-4764-8942-133cded65ce5\") " pod="openshift-ingress/e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8"
Sep 09 16:25:04.925196 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:04.925156    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbqrs\" (UniqueName: \"kubernetes.io/projected/cf6e0304-8dce-4764-8942-133cded65ce5-kube-api-access-cbqrs\") pod \"e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8\" (UID: \"cf6e0304-8dce-4764-8942-133cded65ce5\") " pod="openshift-ingress/e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8"
Sep 09 16:25:04.925196 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:04.925202    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf6e0304-8dce-4764-8942-133cded65ce5-istio-envoy\") pod \"e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8\" (UID: \"cf6e0304-8dce-4764-8942-133cded65ce5\") " pod="openshift-ingress/e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8"
Sep 09 16:25:04.925417 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:04.925227    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf6e0304-8dce-4764-8942-133cded65ce5-istio-token\") pod \"e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8\" (UID: \"cf6e0304-8dce-4764-8942-133cded65ce5\") " pod="openshift-ingress/e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8"
Sep 09 16:25:04.925417 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:04.925341    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf6e0304-8dce-4764-8942-133cded65ce5-workload-certs\") pod \"e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8\" (UID: \"cf6e0304-8dce-4764-8942-133cded65ce5\") " pod="openshift-ingress/e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8"
Sep 09 16:25:04.925417 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:04.925383    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf6e0304-8dce-4764-8942-133cded65ce5-istio-data\") pod \"e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8\" (UID: \"cf6e0304-8dce-4764-8942-133cded65ce5\") " pod="openshift-ingress/e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8"
Sep 09 16:25:04.925645 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:04.925441    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf6e0304-8dce-4764-8942-133cded65ce5-workload-socket\") pod \"e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8\" (UID: \"cf6e0304-8dce-4764-8942-133cded65ce5\") " pod="openshift-ingress/e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8"
Sep 09 16:25:04.925645 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:04.925560    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf6e0304-8dce-4764-8942-133cded65ce5-credential-socket\") pod \"e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8\" (UID: \"cf6e0304-8dce-4764-8942-133cded65ce5\") " pod="openshift-ingress/e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8"
Sep 09 16:25:04.925645 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:04.925599    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf6e0304-8dce-4764-8942-133cded65ce5-istio-podinfo\") pod \"e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8\" (UID: \"cf6e0304-8dce-4764-8942-133cded65ce5\") " pod="openshift-ingress/e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8"
Sep 09 16:25:04.925645 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:04.925636    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf6e0304-8dce-4764-8942-133cded65ce5-istiod-ca-cert\") pod \"e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8\" (UID: \"cf6e0304-8dce-4764-8942-133cded65ce5\") " pod="openshift-ingress/e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8"
Sep 09 16:25:04.925836 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:04.925758    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf6e0304-8dce-4764-8942-133cded65ce5-workload-certs\") pod \"e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8\" (UID: \"cf6e0304-8dce-4764-8942-133cded65ce5\") " pod="openshift-ingress/e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8"
Sep 09 16:25:04.925836 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:04.925800    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf6e0304-8dce-4764-8942-133cded65ce5-istio-data\") pod \"e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8\" (UID: \"cf6e0304-8dce-4764-8942-133cded65ce5\") " pod="openshift-ingress/e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8"
Sep 09 16:25:04.925944 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:04.925914    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf6e0304-8dce-4764-8942-133cded65ce5-workload-socket\") pod \"e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8\" (UID: \"cf6e0304-8dce-4764-8942-133cded65ce5\") " pod="openshift-ingress/e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8"
Sep 09 16:25:04.926052 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:04.926028    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf6e0304-8dce-4764-8942-133cded65ce5-credential-socket\") pod \"e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8\" (UID: \"cf6e0304-8dce-4764-8942-133cded65ce5\") " pod="openshift-ingress/e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8"
Sep 09 16:25:04.926292 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:04.926272    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf6e0304-8dce-4764-8942-133cded65ce5-istiod-ca-cert\") pod \"e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8\" (UID: \"cf6e0304-8dce-4764-8942-133cded65ce5\") " pod="openshift-ingress/e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8"
Sep 09 16:25:04.929526 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:04.928300    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf6e0304-8dce-4764-8942-133cded65ce5-istio-envoy\") pod \"e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8\" (UID: \"cf6e0304-8dce-4764-8942-133cded65ce5\") " pod="openshift-ingress/e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8"
Sep 09 16:25:04.929526 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:04.929201    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf6e0304-8dce-4764-8942-133cded65ce5-istio-podinfo\") pod \"e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8\" (UID: \"cf6e0304-8dce-4764-8942-133cded65ce5\") " pod="openshift-ingress/e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8"
Sep 09 16:25:04.933873 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:04.933846    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbqrs\" (UniqueName: \"kubernetes.io/projected/cf6e0304-8dce-4764-8942-133cded65ce5-kube-api-access-cbqrs\") pod \"e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8\" (UID: \"cf6e0304-8dce-4764-8942-133cded65ce5\") " pod="openshift-ingress/e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8"
Sep 09 16:25:04.934099 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:04.934081    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf6e0304-8dce-4764-8942-133cded65ce5-istio-token\") pod \"e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8\" (UID: \"cf6e0304-8dce-4764-8942-133cded65ce5\") " pod="openshift-ingress/e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8"
Sep 09 16:25:05.070185 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:05.070142    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8"
Sep 09 16:25:05.346023 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:05.345978    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8"]
Sep 09 16:25:05.348253 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:25:05.348217    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf6e0304_8dce_4764_8942_133cded65ce5.slice/crio-985de15a0f61f1af7a1051e902cd9fbf80ad9883a054c3cae44a47916d070170 WatchSource:0}: Error finding container 985de15a0f61f1af7a1051e902cd9fbf80ad9883a054c3cae44a47916d070170: Status 404 returned error can't find the container with id 985de15a0f61f1af7a1051e902cd9fbf80ad9883a054c3cae44a47916d070170
Sep 09 16:25:05.350336 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:05.350294    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:25:05.350419 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:05.350361    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:25:05.350419 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:05.350390    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:25:05.888447 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:05.888412    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8" event={"ID":"cf6e0304-8dce-4764-8942-133cded65ce5","Type":"ContainerStarted","Data":"1e65dcf563fa00fe0e055ad3f2b01cfa876c28759f9c710efc4036d0b7e3cbaa"}
Sep 09 16:25:05.888871 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:05.888487    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8" event={"ID":"cf6e0304-8dce-4764-8942-133cded65ce5","Type":"ContainerStarted","Data":"985de15a0f61f1af7a1051e902cd9fbf80ad9883a054c3cae44a47916d070170"}
Sep 09 16:25:05.910637 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:05.910587    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8" podStartSLOduration=1.910570367 podStartE2EDuration="1.910570367s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 16:25:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 16:25:05.908730896 +0000 UTC m=+981.326580895" watchObservedRunningTime="2026-09-09 16:25:05.910570367 +0000 UTC m=+981.328420366"
Sep 09 16:25:06.070548 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:06.070466    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8"
Sep 09 16:25:06.075237 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:06.075214    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8"
Sep 09 16:25:06.896798 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:06.896756    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8"
Sep 09 16:25:06.902019 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:06.901974    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8"
Sep 09 16:25:08.213489 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:08.212771    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-autores-743aed-openshift-default-658d874dc4-lzxsj"]
Sep 09 16:25:08.225702 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:08.224474    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-743aed-openshift-default-658d874dc4-lzxsj"
Sep 09 16:25:08.227583 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:08.227560    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-autores-743aed-openshift-default-dockercfg-nbd9v\""
Sep 09 16:25:08.236489 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:08.232650    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-743aed-openshift-default-658d874dc4-lzxsj"]
Sep 09 16:25:08.360046 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:08.360010    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b6f1200-474b-4101-b718-2b2d0cdd490a-credential-socket\") pod \"e2e-autores-743aed-openshift-default-658d874dc4-lzxsj\" (UID: \"2b6f1200-474b-4101-b718-2b2d0cdd490a\") " pod="openshift-ingress/e2e-autores-743aed-openshift-default-658d874dc4-lzxsj"
Sep 09 16:25:08.360228 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:08.360066    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b6f1200-474b-4101-b718-2b2d0cdd490a-istio-podinfo\") pod \"e2e-autores-743aed-openshift-default-658d874dc4-lzxsj\" (UID: \"2b6f1200-474b-4101-b718-2b2d0cdd490a\") " pod="openshift-ingress/e2e-autores-743aed-openshift-default-658d874dc4-lzxsj"
Sep 09 16:25:08.360228 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:08.360085    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b6f1200-474b-4101-b718-2b2d0cdd490a-istiod-ca-cert\") pod \"e2e-autores-743aed-openshift-default-658d874dc4-lzxsj\" (UID: \"2b6f1200-474b-4101-b718-2b2d0cdd490a\") " pod="openshift-ingress/e2e-autores-743aed-openshift-default-658d874dc4-lzxsj"
Sep 09 16:25:08.360300 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:08.360229    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b6f1200-474b-4101-b718-2b2d0cdd490a-istio-data\") pod \"e2e-autores-743aed-openshift-default-658d874dc4-lzxsj\" (UID: \"2b6f1200-474b-4101-b718-2b2d0cdd490a\") " pod="openshift-ingress/e2e-autores-743aed-openshift-default-658d874dc4-lzxsj"
Sep 09 16:25:08.360300 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:08.360270    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b6f1200-474b-4101-b718-2b2d0cdd490a-workload-certs\") pod \"e2e-autores-743aed-openshift-default-658d874dc4-lzxsj\" (UID: \"2b6f1200-474b-4101-b718-2b2d0cdd490a\") " pod="openshift-ingress/e2e-autores-743aed-openshift-default-658d874dc4-lzxsj"
Sep 09 16:25:08.360300 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:08.360287    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b6f1200-474b-4101-b718-2b2d0cdd490a-istio-envoy\") pod \"e2e-autores-743aed-openshift-default-658d874dc4-lzxsj\" (UID: \"2b6f1200-474b-4101-b718-2b2d0cdd490a\") " pod="openshift-ingress/e2e-autores-743aed-openshift-default-658d874dc4-lzxsj"
Sep 09 16:25:08.360390 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:08.360329    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b6f1200-474b-4101-b718-2b2d0cdd490a-istio-token\") pod \"e2e-autores-743aed-openshift-default-658d874dc4-lzxsj\" (UID: \"2b6f1200-474b-4101-b718-2b2d0cdd490a\") " pod="openshift-ingress/e2e-autores-743aed-openshift-default-658d874dc4-lzxsj"
Sep 09 16:25:08.360390 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:08.360354    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b6f1200-474b-4101-b718-2b2d0cdd490a-workload-socket\") pod \"e2e-autores-743aed-openshift-default-658d874dc4-lzxsj\" (UID: \"2b6f1200-474b-4101-b718-2b2d0cdd490a\") " pod="openshift-ingress/e2e-autores-743aed-openshift-default-658d874dc4-lzxsj"
Sep 09 16:25:08.360390 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:08.360373    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2nd6\" (UniqueName: \"kubernetes.io/projected/2b6f1200-474b-4101-b718-2b2d0cdd490a-kube-api-access-r2nd6\") pod \"e2e-autores-743aed-openshift-default-658d874dc4-lzxsj\" (UID: \"2b6f1200-474b-4101-b718-2b2d0cdd490a\") " pod="openshift-ingress/e2e-autores-743aed-openshift-default-658d874dc4-lzxsj"
Sep 09 16:25:08.461607 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:08.461574    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b6f1200-474b-4101-b718-2b2d0cdd490a-istio-data\") pod \"e2e-autores-743aed-openshift-default-658d874dc4-lzxsj\" (UID: \"2b6f1200-474b-4101-b718-2b2d0cdd490a\") " pod="openshift-ingress/e2e-autores-743aed-openshift-default-658d874dc4-lzxsj"
Sep 09 16:25:08.461794 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:08.461617    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b6f1200-474b-4101-b718-2b2d0cdd490a-workload-certs\") pod \"e2e-autores-743aed-openshift-default-658d874dc4-lzxsj\" (UID: \"2b6f1200-474b-4101-b718-2b2d0cdd490a\") " pod="openshift-ingress/e2e-autores-743aed-openshift-default-658d874dc4-lzxsj"
Sep 09 16:25:08.461794 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:08.461635    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b6f1200-474b-4101-b718-2b2d0cdd490a-istio-envoy\") pod \"e2e-autores-743aed-openshift-default-658d874dc4-lzxsj\" (UID: \"2b6f1200-474b-4101-b718-2b2d0cdd490a\") " pod="openshift-ingress/e2e-autores-743aed-openshift-default-658d874dc4-lzxsj"
Sep 09 16:25:08.461794 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:08.461668    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b6f1200-474b-4101-b718-2b2d0cdd490a-istio-token\") pod \"e2e-autores-743aed-openshift-default-658d874dc4-lzxsj\" (UID: \"2b6f1200-474b-4101-b718-2b2d0cdd490a\") " pod="openshift-ingress/e2e-autores-743aed-openshift-default-658d874dc4-lzxsj"
Sep 09 16:25:08.461794 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:08.461707    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b6f1200-474b-4101-b718-2b2d0cdd490a-workload-socket\") pod \"e2e-autores-743aed-openshift-default-658d874dc4-lzxsj\" (UID: \"2b6f1200-474b-4101-b718-2b2d0cdd490a\") " pod="openshift-ingress/e2e-autores-743aed-openshift-default-658d874dc4-lzxsj"
Sep 09 16:25:08.461794 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:08.461736    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2nd6\" (UniqueName: \"kubernetes.io/projected/2b6f1200-474b-4101-b718-2b2d0cdd490a-kube-api-access-r2nd6\") pod \"e2e-autores-743aed-openshift-default-658d874dc4-lzxsj\" (UID: \"2b6f1200-474b-4101-b718-2b2d0cdd490a\") " pod="openshift-ingress/e2e-autores-743aed-openshift-default-658d874dc4-lzxsj"
Sep 09 16:25:08.461794 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:08.461776    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b6f1200-474b-4101-b718-2b2d0cdd490a-credential-socket\") pod \"e2e-autores-743aed-openshift-default-658d874dc4-lzxsj\" (UID: \"2b6f1200-474b-4101-b718-2b2d0cdd490a\") " pod="openshift-ingress/e2e-autores-743aed-openshift-default-658d874dc4-lzxsj"
Sep 09 16:25:08.462061 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:08.461822    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b6f1200-474b-4101-b718-2b2d0cdd490a-istio-podinfo\") pod \"e2e-autores-743aed-openshift-default-658d874dc4-lzxsj\" (UID: \"2b6f1200-474b-4101-b718-2b2d0cdd490a\") " pod="openshift-ingress/e2e-autores-743aed-openshift-default-658d874dc4-lzxsj"
Sep 09 16:25:08.462061 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:08.461848    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b6f1200-474b-4101-b718-2b2d0cdd490a-istiod-ca-cert\") pod \"e2e-autores-743aed-openshift-default-658d874dc4-lzxsj\" (UID: \"2b6f1200-474b-4101-b718-2b2d0cdd490a\") " pod="openshift-ingress/e2e-autores-743aed-openshift-default-658d874dc4-lzxsj"
Sep 09 16:25:08.462061 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:08.462017    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b6f1200-474b-4101-b718-2b2d0cdd490a-workload-certs\") pod \"e2e-autores-743aed-openshift-default-658d874dc4-lzxsj\" (UID: \"2b6f1200-474b-4101-b718-2b2d0cdd490a\") " pod="openshift-ingress/e2e-autores-743aed-openshift-default-658d874dc4-lzxsj"
Sep 09 16:25:08.462218 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:08.462119    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b6f1200-474b-4101-b718-2b2d0cdd490a-istio-data\") pod \"e2e-autores-743aed-openshift-default-658d874dc4-lzxsj\" (UID: \"2b6f1200-474b-4101-b718-2b2d0cdd490a\") " pod="openshift-ingress/e2e-autores-743aed-openshift-default-658d874dc4-lzxsj"
Sep 09 16:25:08.462297 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:08.462276    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b6f1200-474b-4101-b718-2b2d0cdd490a-workload-socket\") pod \"e2e-autores-743aed-openshift-default-658d874dc4-lzxsj\" (UID: \"2b6f1200-474b-4101-b718-2b2d0cdd490a\") " pod="openshift-ingress/e2e-autores-743aed-openshift-default-658d874dc4-lzxsj"
Sep 09 16:25:08.462649 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:08.462625    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b6f1200-474b-4101-b718-2b2d0cdd490a-credential-socket\") pod \"e2e-autores-743aed-openshift-default-658d874dc4-lzxsj\" (UID: \"2b6f1200-474b-4101-b718-2b2d0cdd490a\") " pod="openshift-ingress/e2e-autores-743aed-openshift-default-658d874dc4-lzxsj"
Sep 09 16:25:08.462925 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:08.462903    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b6f1200-474b-4101-b718-2b2d0cdd490a-istiod-ca-cert\") pod \"e2e-autores-743aed-openshift-default-658d874dc4-lzxsj\" (UID: \"2b6f1200-474b-4101-b718-2b2d0cdd490a\") " pod="openshift-ingress/e2e-autores-743aed-openshift-default-658d874dc4-lzxsj"
Sep 09 16:25:08.465223 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:08.465111    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b6f1200-474b-4101-b718-2b2d0cdd490a-istio-envoy\") pod \"e2e-autores-743aed-openshift-default-658d874dc4-lzxsj\" (UID: \"2b6f1200-474b-4101-b718-2b2d0cdd490a\") " pod="openshift-ingress/e2e-autores-743aed-openshift-default-658d874dc4-lzxsj"
Sep 09 16:25:08.465223 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:08.465130    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b6f1200-474b-4101-b718-2b2d0cdd490a-istio-podinfo\") pod \"e2e-autores-743aed-openshift-default-658d874dc4-lzxsj\" (UID: \"2b6f1200-474b-4101-b718-2b2d0cdd490a\") " pod="openshift-ingress/e2e-autores-743aed-openshift-default-658d874dc4-lzxsj"
Sep 09 16:25:08.476980 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:08.476958    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b6f1200-474b-4101-b718-2b2d0cdd490a-istio-token\") pod \"e2e-autores-743aed-openshift-default-658d874dc4-lzxsj\" (UID: \"2b6f1200-474b-4101-b718-2b2d0cdd490a\") " pod="openshift-ingress/e2e-autores-743aed-openshift-default-658d874dc4-lzxsj"
Sep 09 16:25:08.477421 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:08.477397    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2nd6\" (UniqueName: \"kubernetes.io/projected/2b6f1200-474b-4101-b718-2b2d0cdd490a-kube-api-access-r2nd6\") pod \"e2e-autores-743aed-openshift-default-658d874dc4-lzxsj\" (UID: \"2b6f1200-474b-4101-b718-2b2d0cdd490a\") " pod="openshift-ingress/e2e-autores-743aed-openshift-default-658d874dc4-lzxsj"
Sep 09 16:25:08.543250 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:08.543213    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-743aed-openshift-default-658d874dc4-lzxsj"
Sep 09 16:25:08.849207 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:08.849182    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-743aed-openshift-default-658d874dc4-lzxsj"]
Sep 09 16:25:08.851305 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:25:08.851277    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b6f1200_474b_4101_b718_2b2d0cdd490a.slice/crio-5ee81144b41495f00945f4fafac8c75144d9e6f9036fa9c0cf605de9e2f4d09c WatchSource:0}: Error finding container 5ee81144b41495f00945f4fafac8c75144d9e6f9036fa9c0cf605de9e2f4d09c: Status 404 returned error can't find the container with id 5ee81144b41495f00945f4fafac8c75144d9e6f9036fa9c0cf605de9e2f4d09c
Sep 09 16:25:08.853659 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:08.853625    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:25:08.853748 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:08.853690    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:25:08.853748 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:08.853717    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:25:08.919637 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:08.919597    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-743aed-openshift-default-658d874dc4-lzxsj" event={"ID":"2b6f1200-474b-4101-b718-2b2d0cdd490a","Type":"ContainerStarted","Data":"5ee81144b41495f00945f4fafac8c75144d9e6f9036fa9c0cf605de9e2f4d09c"}
Sep 09 16:25:09.926058 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:09.926017    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-743aed-openshift-default-658d874dc4-lzxsj" event={"ID":"2b6f1200-474b-4101-b718-2b2d0cdd490a","Type":"ContainerStarted","Data":"5d6d5a000288ced75e228c00aa5c4477a894bb9748d4b724422803e7edfe39c4"}
Sep 09 16:25:09.946331 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:09.946280    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-autores-743aed-openshift-default-658d874dc4-lzxsj" podStartSLOduration=1.946263563 podStartE2EDuration="1.946263563s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 16:25:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 16:25:09.943781185 +0000 UTC m=+985.361631184" watchObservedRunningTime="2026-09-09 16:25:09.946263563 +0000 UTC m=+985.364113553"
Sep 09 16:25:10.543525 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:10.543480    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-743aed-openshift-default-658d874dc4-lzxsj"
Sep 09 16:25:10.548543 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:10.548514    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-743aed-openshift-default-658d874dc4-lzxsj"
Sep 09 16:25:10.930830 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:10.930795    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-743aed-openshift-default-658d874dc4-lzxsj"
Sep 09 16:25:10.931708 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:10.931689    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-743aed-openshift-default-658d874dc4-lzxsj"
Sep 09 16:25:11.616528 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:11.616492    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:25:14.895770 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:14.895685    2567 patch_prober.go:28] interesting pod/e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 09 16:25:14.896277 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:14.895832    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl" podUID="f1e36da0-3a7c-4765-aeb6-0b6339a09cc4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 09 16:25:15.959993 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:15.959907    2567 generic.go:356] "Generic (PLEG): container finished" podID="f1e36da0-3a7c-4765-aeb6-0b6339a09cc4" containerID="6ddf2705b7bd4446dc2110af85a6252fde2d8f77b3c9729524b6a6a6c72f880b" exitCode=0
Sep 09 16:25:15.960528 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:15.959990    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl" event={"ID":"f1e36da0-3a7c-4765-aeb6-0b6339a09cc4","Type":"ContainerDied","Data":"6ddf2705b7bd4446dc2110af85a6252fde2d8f77b3c9729524b6a6a6c72f880b"}
Sep 09 16:25:15.960604 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:15.960547    2567 scope.go:117] "RemoveContainer" containerID="6ddf2705b7bd4446dc2110af85a6252fde2d8f77b3c9729524b6a6a6c72f880b"
Sep 09 16:25:15.961706 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:15.961677    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:25:15.961767 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:15.961748    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:25:15.961803 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:15.961780    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:25:16.968494 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:16.968438    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl" event={"ID":"f1e36da0-3a7c-4765-aeb6-0b6339a09cc4","Type":"ContainerStarted","Data":"75a99bdfb20bee38d1ed434e26628356d776f38035872feefd7b50f587a07fe1"}
Sep 09 16:25:17.845095 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:17.845050    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl"
Sep 09 16:25:17.845317 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:17.845187    2567 patch_prober.go:28] interesting pod/e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body=
Sep 09 16:25:17.845317 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:17.845252    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl" podUID="f1e36da0-3a7c-4765-aeb6-0b6339a09cc4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused"
Sep 09 16:25:18.845753 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:18.845715    2567 patch_prober.go:28] interesting pod/e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body=
Sep 09 16:25:18.846232 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:18.845831    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl" podUID="f1e36da0-3a7c-4765-aeb6-0b6339a09cc4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused"
Sep 09 16:25:19.845476 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:19.845422    2567 patch_prober.go:28] interesting pod/e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body=
Sep 09 16:25:19.845651 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:19.845510    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl" podUID="f1e36da0-3a7c-4765-aeb6-0b6339a09cc4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused"
Sep 09 16:25:20.845838 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:20.845796    2567 patch_prober.go:28] interesting pod/e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body=
Sep 09 16:25:20.846261 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:20.845880    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl" podUID="f1e36da0-3a7c-4765-aeb6-0b6339a09cc4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused"
Sep 09 16:25:21.844936 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:21.844901    2567 patch_prober.go:28] interesting pod/e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body=
Sep 09 16:25:21.847109 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:21.845152    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl" podUID="f1e36da0-3a7c-4765-aeb6-0b6339a09cc4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused"
Sep 09 16:25:21.951851 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:21.951812    2567 patch_prober.go:28] interesting pod/e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": read tcp 10.134.0.2:48396->10.134.0.30:15021: read: connection reset by peer" start-of-body=
Sep 09 16:25:21.951988 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:21.951892    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8" podUID="cf6e0304-8dce-4764-8942-133cded65ce5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": read tcp 10.134.0.2:48396->10.134.0.30:15021: read: connection reset by peer"
Sep 09 16:25:21.999429 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:21.999383    2567 generic.go:356] "Generic (PLEG): container finished" podID="cf6e0304-8dce-4764-8942-133cded65ce5" containerID="1e65dcf563fa00fe0e055ad3f2b01cfa876c28759f9c710efc4036d0b7e3cbaa" exitCode=0
Sep 09 16:25:21.999636 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:21.999474    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8" event={"ID":"cf6e0304-8dce-4764-8942-133cded65ce5","Type":"ContainerDied","Data":"1e65dcf563fa00fe0e055ad3f2b01cfa876c28759f9c710efc4036d0b7e3cbaa"}
Sep 09 16:25:21.999989 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:21.999962    2567 scope.go:117] "RemoveContainer" containerID="1e65dcf563fa00fe0e055ad3f2b01cfa876c28759f9c710efc4036d0b7e3cbaa"
Sep 09 16:25:22.001937 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:22.001894    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:25:22.002049 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:22.002022    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:25:22.002124 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:22.002100    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:25:22.850879 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:22.850838    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl"
Sep 09 16:25:22.851529 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:22.851494    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl"
Sep 09 16:25:22.852314 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:22.852293    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl"
Sep 09 16:25:23.017480 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:23.016719    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8" event={"ID":"cf6e0304-8dce-4764-8942-133cded65ce5","Type":"ContainerStarted","Data":"7dbd236da2d26ef0f61784571b83ed327d0325f7d5efd5707873822fc9b57885"}
Sep 09 16:25:23.070919 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:23.070885    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8"
Sep 09 16:25:23.071106 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:23.071082    2567 patch_prober.go:28] interesting pod/e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body=
Sep 09 16:25:23.071202 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:23.071149    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8" podUID="cf6e0304-8dce-4764-8942-133cded65ce5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused"
Sep 09 16:25:24.070725 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:24.070686    2567 patch_prober.go:28] interesting pod/e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body=
Sep 09 16:25:24.071188 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:24.070739    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8" podUID="cf6e0304-8dce-4764-8942-133cded65ce5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused"
Sep 09 16:25:25.070680 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:25.070641    2567 patch_prober.go:28] interesting pod/e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body=
Sep 09 16:25:25.070862 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:25.070718    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8" podUID="cf6e0304-8dce-4764-8942-133cded65ce5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused"
Sep 09 16:25:25.625485 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:25.623003    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc"]
Sep 09 16:25:25.631483 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:25.628895    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc"
Sep 09 16:25:25.641481 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:25.633772    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-c23332-openshift-default-dockercfg-7jb46\""
Sep 09 16:25:25.654489 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:25.652725    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc"]
Sep 09 16:25:25.684873 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:25.684840    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/155986b3-9283-4eea-bff1-479cad0f546f-istio-envoy\") pod \"e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc\" (UID: \"155986b3-9283-4eea-bff1-479cad0f546f\") " pod="openshift-ingress/e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc"
Sep 09 16:25:25.685034 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:25.684905    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/155986b3-9283-4eea-bff1-479cad0f546f-istio-token\") pod \"e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc\" (UID: \"155986b3-9283-4eea-bff1-479cad0f546f\") " pod="openshift-ingress/e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc"
Sep 09 16:25:25.685034 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:25.684941    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/155986b3-9283-4eea-bff1-479cad0f546f-workload-certs\") pod \"e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc\" (UID: \"155986b3-9283-4eea-bff1-479cad0f546f\") " pod="openshift-ingress/e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc"
Sep 09 16:25:25.685034 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:25.684973    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/155986b3-9283-4eea-bff1-479cad0f546f-workload-socket\") pod \"e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc\" (UID: \"155986b3-9283-4eea-bff1-479cad0f546f\") " pod="openshift-ingress/e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc"
Sep 09 16:25:25.685034 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:25.685008    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wcs9\" (UniqueName: \"kubernetes.io/projected/155986b3-9283-4eea-bff1-479cad0f546f-kube-api-access-5wcs9\") pod \"e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc\" (UID: \"155986b3-9283-4eea-bff1-479cad0f546f\") " pod="openshift-ingress/e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc"
Sep 09 16:25:25.685171 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:25.685052    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/155986b3-9283-4eea-bff1-479cad0f546f-istiod-ca-cert\") pod \"e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc\" (UID: \"155986b3-9283-4eea-bff1-479cad0f546f\") " pod="openshift-ingress/e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc"
Sep 09 16:25:25.685171 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:25.685078    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/155986b3-9283-4eea-bff1-479cad0f546f-istio-podinfo\") pod \"e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc\" (UID: \"155986b3-9283-4eea-bff1-479cad0f546f\") " pod="openshift-ingress/e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc"
Sep 09 16:25:25.685171 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:25.685097    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/155986b3-9283-4eea-bff1-479cad0f546f-istio-data\") pod \"e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc\" (UID: \"155986b3-9283-4eea-bff1-479cad0f546f\") " pod="openshift-ingress/e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc"
Sep 09 16:25:25.685171 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:25.685123    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/155986b3-9283-4eea-bff1-479cad0f546f-credential-socket\") pod \"e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc\" (UID: \"155986b3-9283-4eea-bff1-479cad0f546f\") " pod="openshift-ingress/e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc"
Sep 09 16:25:25.786680 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:25.786640    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/155986b3-9283-4eea-bff1-479cad0f546f-istio-podinfo\") pod \"e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc\" (UID: \"155986b3-9283-4eea-bff1-479cad0f546f\") " pod="openshift-ingress/e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc"
Sep 09 16:25:25.786850 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:25.786690    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/155986b3-9283-4eea-bff1-479cad0f546f-istio-data\") pod \"e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc\" (UID: \"155986b3-9283-4eea-bff1-479cad0f546f\") " pod="openshift-ingress/e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc"
Sep 09 16:25:25.786850 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:25.786720    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/155986b3-9283-4eea-bff1-479cad0f546f-credential-socket\") pod \"e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc\" (UID: \"155986b3-9283-4eea-bff1-479cad0f546f\") " pod="openshift-ingress/e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc"
Sep 09 16:25:25.786850 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:25.786768    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/155986b3-9283-4eea-bff1-479cad0f546f-istio-envoy\") pod \"e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc\" (UID: \"155986b3-9283-4eea-bff1-479cad0f546f\") " pod="openshift-ingress/e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc"
Sep 09 16:25:25.786850 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:25.786817    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/155986b3-9283-4eea-bff1-479cad0f546f-istio-token\") pod \"e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc\" (UID: \"155986b3-9283-4eea-bff1-479cad0f546f\") " pod="openshift-ingress/e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc"
Sep 09 16:25:25.786850 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:25.786849    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/155986b3-9283-4eea-bff1-479cad0f546f-workload-certs\") pod \"e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc\" (UID: \"155986b3-9283-4eea-bff1-479cad0f546f\") " pod="openshift-ingress/e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc"
Sep 09 16:25:25.787020 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:25.786878    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/155986b3-9283-4eea-bff1-479cad0f546f-workload-socket\") pod \"e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc\" (UID: \"155986b3-9283-4eea-bff1-479cad0f546f\") " pod="openshift-ingress/e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc"
Sep 09 16:25:25.787020 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:25.786906    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wcs9\" (UniqueName: \"kubernetes.io/projected/155986b3-9283-4eea-bff1-479cad0f546f-kube-api-access-5wcs9\") pod \"e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc\" (UID: \"155986b3-9283-4eea-bff1-479cad0f546f\") " pod="openshift-ingress/e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc"
Sep 09 16:25:25.787020 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:25.786928    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/155986b3-9283-4eea-bff1-479cad0f546f-istiod-ca-cert\") pod \"e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc\" (UID: \"155986b3-9283-4eea-bff1-479cad0f546f\") " pod="openshift-ingress/e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc"
Sep 09 16:25:25.787653 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:25.787633    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/155986b3-9283-4eea-bff1-479cad0f546f-istiod-ca-cert\") pod \"e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc\" (UID: \"155986b3-9283-4eea-bff1-479cad0f546f\") " pod="openshift-ingress/e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc"
Sep 09 16:25:25.788310 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:25.788283    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/155986b3-9283-4eea-bff1-479cad0f546f-credential-socket\") pod \"e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc\" (UID: \"155986b3-9283-4eea-bff1-479cad0f546f\") " pod="openshift-ingress/e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc"
Sep 09 16:25:25.788485 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:25.788366    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/155986b3-9283-4eea-bff1-479cad0f546f-workload-certs\") pod \"e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc\" (UID: \"155986b3-9283-4eea-bff1-479cad0f546f\") " pod="openshift-ingress/e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc"
Sep 09 16:25:25.788591 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:25.788384    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/155986b3-9283-4eea-bff1-479cad0f546f-workload-socket\") pod \"e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc\" (UID: \"155986b3-9283-4eea-bff1-479cad0f546f\") " pod="openshift-ingress/e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc"
Sep 09 16:25:25.788705 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:25.788684    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/155986b3-9283-4eea-bff1-479cad0f546f-istio-data\") pod \"e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc\" (UID: \"155986b3-9283-4eea-bff1-479cad0f546f\") " pod="openshift-ingress/e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc"
Sep 09 16:25:25.793781 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:25.793757    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/155986b3-9283-4eea-bff1-479cad0f546f-istio-envoy\") pod \"e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc\" (UID: \"155986b3-9283-4eea-bff1-479cad0f546f\") " pod="openshift-ingress/e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc"
Sep 09 16:25:25.796139 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:25.793995    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/155986b3-9283-4eea-bff1-479cad0f546f-istio-podinfo\") pod \"e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc\" (UID: \"155986b3-9283-4eea-bff1-479cad0f546f\") " pod="openshift-ingress/e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc"
Sep 09 16:25:25.803235 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:25.803176    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wcs9\" (UniqueName: \"kubernetes.io/projected/155986b3-9283-4eea-bff1-479cad0f546f-kube-api-access-5wcs9\") pod \"e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc\" (UID: \"155986b3-9283-4eea-bff1-479cad0f546f\") " pod="openshift-ingress/e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc"
Sep 09 16:25:25.803808 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:25.803784    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/155986b3-9283-4eea-bff1-479cad0f546f-istio-token\") pod \"e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc\" (UID: \"155986b3-9283-4eea-bff1-479cad0f546f\") " pod="openshift-ingress/e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc"
Sep 09 16:25:25.957868 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:25.957644    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc"
Sep 09 16:25:26.071177 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:26.071106    2567 patch_prober.go:28] interesting pod/e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body=
Sep 09 16:25:26.071567 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:26.071178    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8" podUID="cf6e0304-8dce-4764-8942-133cded65ce5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused"
Sep 09 16:25:26.292550 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:26.292314    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc"]
Sep 09 16:25:26.306214 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:26.306158    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:25:26.306336 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:26.306263    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:25:26.306336 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:26.306309    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:25:27.051961 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:27.051871    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc" event={"ID":"155986b3-9283-4eea-bff1-479cad0f546f","Type":"ContainerStarted","Data":"96fc12418897c31e2d4ce3ebed36ff24c906ae77699a6aed090a6dc1676f41b2"}
Sep 09 16:25:27.051961 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:27.051933    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc" event={"ID":"155986b3-9283-4eea-bff1-479cad0f546f","Type":"ContainerStarted","Data":"e6d5585416941cfe24360e4a92eb868f53a48d3a164b0e6e8fdfa11a3fdc521d"}
Sep 09 16:25:27.070621 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:27.070587    2567 patch_prober.go:28] interesting pod/e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body=
Sep 09 16:25:27.071253 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:27.070865    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8" podUID="cf6e0304-8dce-4764-8942-133cded65ce5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused"
Sep 09 16:25:27.071799 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:27.071761    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc" podStartSLOduration=2.071746602 podStartE2EDuration="2.071746602s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 16:25:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 16:25:27.070930744 +0000 UTC m=+1002.488780768" watchObservedRunningTime="2026-09-09 16:25:27.071746602 +0000 UTC m=+1002.489596601"
Sep 09 16:25:27.958771 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:27.958681    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc"
Sep 09 16:25:27.964997 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:27.964966    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc"
Sep 09 16:25:28.056696 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:28.056661    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc"
Sep 09 16:25:28.057958 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:28.057937    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc"
Sep 09 16:25:28.071164 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:28.071132    2567 patch_prober.go:28] interesting pod/e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body=
Sep 09 16:25:28.071286 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:28.071185    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8" podUID="cf6e0304-8dce-4764-8942-133cded65ce5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused"
Sep 09 16:25:28.805114 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:28.805074    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:25:29.074834 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:29.074809    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8"
Sep 09 16:25:29.075216 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:29.075100    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8"
Sep 09 16:25:29.075947 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:29.075927    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8"
Sep 09 16:25:31.071796 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:31.071761    2567 generic.go:356] "Generic (PLEG): container finished" podID="2b6f1200-474b-4101-b718-2b2d0cdd490a" containerID="5d6d5a000288ced75e228c00aa5c4477a894bb9748d4b724422803e7edfe39c4" exitCode=0
Sep 09 16:25:31.072221 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:31.071836    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-743aed-openshift-default-658d874dc4-lzxsj" event={"ID":"2b6f1200-474b-4101-b718-2b2d0cdd490a","Type":"ContainerDied","Data":"5d6d5a000288ced75e228c00aa5c4477a894bb9748d4b724422803e7edfe39c4"}
Sep 09 16:25:31.072353 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:31.072338    2567 scope.go:117] "RemoveContainer" containerID="5d6d5a000288ced75e228c00aa5c4477a894bb9748d4b724422803e7edfe39c4"
Sep 09 16:25:31.073724 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:31.073689    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:25:31.073837 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:31.073771    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:25:31.073837 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:31.073807    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:25:32.078978 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:32.078941    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-743aed-openshift-default-658d874dc4-lzxsj" event={"ID":"2b6f1200-474b-4101-b718-2b2d0cdd490a","Type":"ContainerStarted","Data":"7f1d5470d42639ebd84df7dd2a9044de67d9eaac7124a4f43e899b86c237895f"}
Sep 09 16:25:32.543798 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:32.543708    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-743aed-openshift-default-658d874dc4-lzxsj"
Sep 09 16:25:32.543956 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:32.543935    2567 patch_prober.go:28] interesting pod/e2e-autores-743aed-openshift-default-658d874dc4-lzxsj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Sep 09 16:25:32.544012 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:32.543991    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-743aed-openshift-default-658d874dc4-lzxsj" podUID="2b6f1200-474b-4101-b718-2b2d0cdd490a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Sep 09 16:25:33.026410 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:33.026374    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0d2e38e1-x9xwv"]
Sep 09 16:25:33.029977 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:33.029950    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0d2e38e1-x9xwv"
Sep 09 16:25:33.032124 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:33.032101    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wxghs\""
Sep 09 16:25:33.032234 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:33.032125    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 09 16:25:33.037724 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:33.037669    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0d2e38e1-x9xwv"]
Sep 09 16:25:33.100050 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:33.100003    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/48074e07-e45a-456c-bbab-ff849b31c899-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0d2e38e1-x9xwv\" (UID: \"48074e07-e45a-456c-bbab-ff849b31c899\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0d2e38e1-x9xwv"
Sep 09 16:25:33.201384 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:33.201332    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/48074e07-e45a-456c-bbab-ff849b31c899-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0d2e38e1-x9xwv\" (UID: \"48074e07-e45a-456c-bbab-ff849b31c899\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0d2e38e1-x9xwv"
Sep 09 16:25:33.203038 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:33.203004    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/48074e07-e45a-456c-bbab-ff849b31c899-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0d2e38e1-x9xwv\" (UID: \"48074e07-e45a-456c-bbab-ff849b31c899\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0d2e38e1-x9xwv"
Sep 09 16:25:33.342175 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:33.342138    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0d2e38e1-x9xwv"
Sep 09 16:25:33.543768 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:33.543728    2567 patch_prober.go:28] interesting pod/e2e-autores-743aed-openshift-default-658d874dc4-lzxsj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Sep 09 16:25:33.543926 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:33.543790    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-743aed-openshift-default-658d874dc4-lzxsj" podUID="2b6f1200-474b-4101-b718-2b2d0cdd490a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Sep 09 16:25:33.633031 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:33.633004    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0d2e38e1-x9xwv"]
Sep 09 16:25:33.635096 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:25:33.635069    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48074e07_e45a_456c_bbab_ff849b31c899.slice/crio-4d8420351d5caeac0d4dd354063dedc0ee0535cf7b1badf22d30a2dce21b22f5 WatchSource:0}: Error finding container 4d8420351d5caeac0d4dd354063dedc0ee0535cf7b1badf22d30a2dce21b22f5: Status 404 returned error can't find the container with id 4d8420351d5caeac0d4dd354063dedc0ee0535cf7b1badf22d30a2dce21b22f5
Sep 09 16:25:33.637204 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:33.637181    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 16:25:34.091429 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:34.091386    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0d2e38e1-x9xwv" event={"ID":"48074e07-e45a-456c-bbab-ff849b31c899","Type":"ContainerStarted","Data":"4d8420351d5caeac0d4dd354063dedc0ee0535cf7b1badf22d30a2dce21b22f5"}
Sep 09 16:25:34.544140 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:34.544049    2567 patch_prober.go:28] interesting pod/e2e-autores-743aed-openshift-default-658d874dc4-lzxsj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Sep 09 16:25:34.544140 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:34.544112    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-743aed-openshift-default-658d874dc4-lzxsj" podUID="2b6f1200-474b-4101-b718-2b2d0cdd490a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Sep 09 16:25:35.544640 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:35.544597    2567 patch_prober.go:28] interesting pod/e2e-autores-743aed-openshift-default-658d874dc4-lzxsj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Sep 09 16:25:35.545048 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:35.544666    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-743aed-openshift-default-658d874dc4-lzxsj" podUID="2b6f1200-474b-4101-b718-2b2d0cdd490a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Sep 09 16:25:36.543915 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:36.543876    2567 patch_prober.go:28] interesting pod/e2e-autores-743aed-openshift-default-658d874dc4-lzxsj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Sep 09 16:25:36.544115 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:36.543946    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-743aed-openshift-default-658d874dc4-lzxsj" podUID="2b6f1200-474b-4101-b718-2b2d0cdd490a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Sep 09 16:25:37.544403 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:37.544365    2567 patch_prober.go:28] interesting pod/e2e-autores-743aed-openshift-default-658d874dc4-lzxsj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Sep 09 16:25:37.544827 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:37.544428    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-743aed-openshift-default-658d874dc4-lzxsj" podUID="2b6f1200-474b-4101-b718-2b2d0cdd490a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Sep 09 16:25:38.131586 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:38.131550    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0d2e38e1-x9xwv" event={"ID":"48074e07-e45a-456c-bbab-ff849b31c899","Type":"ContainerStarted","Data":"5a7f3da675cc8ea7ec5087b778301c19f38462c0b6054f9e0b62b1c8be0246ff"}
Sep 09 16:25:38.152303 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:38.152248    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0d2e38e1-x9xwv" podStartSLOduration=0.868230597 podStartE2EDuration="5.152231212s" totalImagesPullingTime="4.284000615s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 16:25:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 16:25:38.149724458 +0000 UTC m=+1013.567574497" watchObservedRunningTime="2026-09-09 16:25:38.152231212 +0000 UTC m=+1013.570081211"
Sep 09 16:25:38.548337 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:38.548308    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-743aed-openshift-default-658d874dc4-lzxsj"
Sep 09 16:25:38.548779 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:38.548651    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-743aed-openshift-default-658d874dc4-lzxsj"
Sep 09 16:25:38.549269 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:38.549252    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-743aed-openshift-default-658d874dc4-lzxsj"
Sep 09 16:25:39.138904 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:39.138855    2567 generic.go:356] "Generic (PLEG): container finished" podID="48074e07-e45a-456c-bbab-ff849b31c899" containerID="5a7f3da675cc8ea7ec5087b778301c19f38462c0b6054f9e0b62b1c8be0246ff" exitCode=0
Sep 09 16:25:39.139087 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:39.138938    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0d2e38e1-x9xwv" event={"ID":"48074e07-e45a-456c-bbab-ff849b31c899","Type":"ContainerDied","Data":"5a7f3da675cc8ea7ec5087b778301c19f38462c0b6054f9e0b62b1c8be0246ff"}
Sep 09 16:25:40.327100 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:40.327072    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0d2e38e1-x9xwv"
Sep 09 16:25:40.484761 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:40.484664    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/48074e07-e45a-456c-bbab-ff849b31c899-maas-api-service-ca\") pod \"48074e07-e45a-456c-bbab-ff849b31c899\" (UID: \"48074e07-e45a-456c-bbab-ff849b31c899\") "
Sep 09 16:25:40.485045 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:40.485020    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48074e07-e45a-456c-bbab-ff849b31c899-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "48074e07-e45a-456c-bbab-ff849b31c899" (UID: "48074e07-e45a-456c-bbab-ff849b31c899"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 16:25:40.585953 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:40.585914    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/48074e07-e45a-456c-bbab-ff849b31c899-maas-api-service-ca\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:25:41.149689 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:41.149652    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0d2e38e1-x9xwv" event={"ID":"48074e07-e45a-456c-bbab-ff849b31c899","Type":"ContainerDied","Data":"4d8420351d5caeac0d4dd354063dedc0ee0535cf7b1badf22d30a2dce21b22f5"}
Sep 09 16:25:41.149689 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:41.149678    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0d2e38e1-x9xwv"
Sep 09 16:25:41.149901 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:41.149707    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4d8420351d5caeac0d4dd354063dedc0ee0535cf7b1badf22d30a2dce21b22f5"
Sep 09 16:25:44.409344 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:44.409303    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:25:48.189863 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:48.189771    2567 generic.go:356] "Generic (PLEG): container finished" podID="155986b3-9283-4eea-bff1-479cad0f546f" containerID="96fc12418897c31e2d4ce3ebed36ff24c906ae77699a6aed090a6dc1676f41b2" exitCode=0
Sep 09 16:25:48.189863 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:48.189852    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc" event={"ID":"155986b3-9283-4eea-bff1-479cad0f546f","Type":"ContainerDied","Data":"96fc12418897c31e2d4ce3ebed36ff24c906ae77699a6aed090a6dc1676f41b2"}
Sep 09 16:25:48.190303 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:48.190253    2567 scope.go:117] "RemoveContainer" containerID="96fc12418897c31e2d4ce3ebed36ff24c906ae77699a6aed090a6dc1676f41b2"
Sep 09 16:25:48.191390 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:48.191361    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:25:48.191442 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:48.191432    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:25:48.191499 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:48.191481    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:25:49.200198 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:49.200152    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc" event={"ID":"155986b3-9283-4eea-bff1-479cad0f546f","Type":"ContainerStarted","Data":"908d3524f14cfafda38ea73492182cda3b7bf663ce8ba126225eb3b43daac779"}
Sep 09 16:25:49.441185 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:49.441144    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0d2e38e1-x9xwv"]
Sep 09 16:25:49.446102 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:49.446073    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0d2e38e1-x9xwv"]
Sep 09 16:25:49.622154 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:49.622113    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8"]
Sep 09 16:25:49.622694 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:49.622655    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8" podUID="cf6e0304-8dce-4764-8942-133cded65ce5" containerName="istio-proxy" containerID="cri-o://7dbd236da2d26ef0f61784571b83ed327d0325f7d5efd5707873822fc9b57885" gracePeriod=30
Sep 09 16:25:49.958897 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:49.958805    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc"
Sep 09 16:25:49.959044 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:49.959002    2567 patch_prober.go:28] interesting pod/e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Sep 09 16:25:49.959086 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:49.959062    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc" podUID="155986b3-9283-4eea-bff1-479cad0f546f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Sep 09 16:25:50.775204 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:50.775165    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm"]
Sep 09 16:25:50.775951 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:50.775928    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48074e07-e45a-456c-bbab-ff849b31c899" containerName="revoke-keys"
Sep 09 16:25:50.775951 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:50.775953    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="48074e07-e45a-456c-bbab-ff849b31c899" containerName="revoke-keys"
Sep 09 16:25:50.776112 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:50.776069    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="48074e07-e45a-456c-bbab-ff849b31c899" containerName="revoke-keys"
Sep 09 16:25:50.779328 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:50.779302    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm"
Sep 09 16:25:50.781876 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:50.781853    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-fb79f974-gw-openshift-default-dockercfg-tl56d\""
Sep 09 16:25:50.795884 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:50.791081    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm"]
Sep 09 16:25:50.797041 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:50.797009    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5749d6e6-548c-40af-928a-7f8471c7492f-istio-envoy\") pod \"e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm\" (UID: \"5749d6e6-548c-40af-928a-7f8471c7492f\") " pod="openshift-ingress/e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm"
Sep 09 16:25:50.797170 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:50.797065    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5749d6e6-548c-40af-928a-7f8471c7492f-workload-socket\") pod \"e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm\" (UID: \"5749d6e6-548c-40af-928a-7f8471c7492f\") " pod="openshift-ingress/e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm"
Sep 09 16:25:50.797170 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:50.797095    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5749d6e6-548c-40af-928a-7f8471c7492f-istiod-ca-cert\") pod \"e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm\" (UID: \"5749d6e6-548c-40af-928a-7f8471c7492f\") " pod="openshift-ingress/e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm"
Sep 09 16:25:50.797170 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:50.797121    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5749d6e6-548c-40af-928a-7f8471c7492f-istio-token\") pod \"e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm\" (UID: \"5749d6e6-548c-40af-928a-7f8471c7492f\") " pod="openshift-ingress/e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm"
Sep 09 16:25:50.797170 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:50.797152    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5749d6e6-548c-40af-928a-7f8471c7492f-credential-socket\") pod \"e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm\" (UID: \"5749d6e6-548c-40af-928a-7f8471c7492f\") " pod="openshift-ingress/e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm"
Sep 09 16:25:50.797397 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:50.797216    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5749d6e6-548c-40af-928a-7f8471c7492f-istio-podinfo\") pod \"e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm\" (UID: \"5749d6e6-548c-40af-928a-7f8471c7492f\") " pod="openshift-ingress/e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm"
Sep 09 16:25:50.797397 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:50.797270    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5749d6e6-548c-40af-928a-7f8471c7492f-workload-certs\") pod \"e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm\" (UID: \"5749d6e6-548c-40af-928a-7f8471c7492f\") " pod="openshift-ingress/e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm"
Sep 09 16:25:50.797397 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:50.797302    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5749d6e6-548c-40af-928a-7f8471c7492f-istio-data\") pod \"e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm\" (UID: \"5749d6e6-548c-40af-928a-7f8471c7492f\") " pod="openshift-ingress/e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm"
Sep 09 16:25:50.798238 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:50.797349    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8zb4\" (UniqueName: \"kubernetes.io/projected/5749d6e6-548c-40af-928a-7f8471c7492f-kube-api-access-m8zb4\") pod \"e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm\" (UID: \"5749d6e6-548c-40af-928a-7f8471c7492f\") " pod="openshift-ingress/e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm"
Sep 09 16:25:50.906079 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:50.906034    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5749d6e6-548c-40af-928a-7f8471c7492f-istio-podinfo\") pod \"e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm\" (UID: \"5749d6e6-548c-40af-928a-7f8471c7492f\") " pod="openshift-ingress/e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm"
Sep 09 16:25:50.906914 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:50.906890    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5749d6e6-548c-40af-928a-7f8471c7492f-workload-certs\") pod \"e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm\" (UID: \"5749d6e6-548c-40af-928a-7f8471c7492f\") " pod="openshift-ingress/e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm"
Sep 09 16:25:50.907093 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:50.907077    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5749d6e6-548c-40af-928a-7f8471c7492f-istio-data\") pod \"e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm\" (UID: \"5749d6e6-548c-40af-928a-7f8471c7492f\") " pod="openshift-ingress/e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm"
Sep 09 16:25:50.907237 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:50.907220    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8zb4\" (UniqueName: \"kubernetes.io/projected/5749d6e6-548c-40af-928a-7f8471c7492f-kube-api-access-m8zb4\") pod \"e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm\" (UID: \"5749d6e6-548c-40af-928a-7f8471c7492f\") " pod="openshift-ingress/e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm"
Sep 09 16:25:50.907367 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:50.907351    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5749d6e6-548c-40af-928a-7f8471c7492f-istio-envoy\") pod \"e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm\" (UID: \"5749d6e6-548c-40af-928a-7f8471c7492f\") " pod="openshift-ingress/e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm"
Sep 09 16:25:50.907512 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:50.907496    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5749d6e6-548c-40af-928a-7f8471c7492f-workload-socket\") pod \"e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm\" (UID: \"5749d6e6-548c-40af-928a-7f8471c7492f\") " pod="openshift-ingress/e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm"
Sep 09 16:25:50.908076 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:50.908052    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5749d6e6-548c-40af-928a-7f8471c7492f-istiod-ca-cert\") pod \"e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm\" (UID: \"5749d6e6-548c-40af-928a-7f8471c7492f\") " pod="openshift-ingress/e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm"
Sep 09 16:25:50.908282 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:50.908266    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5749d6e6-548c-40af-928a-7f8471c7492f-istio-token\") pod \"e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm\" (UID: \"5749d6e6-548c-40af-928a-7f8471c7492f\") " pod="openshift-ingress/e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm"
Sep 09 16:25:50.908414 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:50.908398    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5749d6e6-548c-40af-928a-7f8471c7492f-credential-socket\") pod \"e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm\" (UID: \"5749d6e6-548c-40af-928a-7f8471c7492f\") " pod="openshift-ingress/e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm"
Sep 09 16:25:50.908925 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:50.908906    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5749d6e6-548c-40af-928a-7f8471c7492f-credential-socket\") pod \"e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm\" (UID: \"5749d6e6-548c-40af-928a-7f8471c7492f\") " pod="openshift-ingress/e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm"
Sep 09 16:25:50.909303 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:50.909285    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5749d6e6-548c-40af-928a-7f8471c7492f-workload-certs\") pod \"e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm\" (UID: \"5749d6e6-548c-40af-928a-7f8471c7492f\") " pod="openshift-ingress/e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm"
Sep 09 16:25:50.909675 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:50.909654    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5749d6e6-548c-40af-928a-7f8471c7492f-istio-data\") pod \"e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm\" (UID: \"5749d6e6-548c-40af-928a-7f8471c7492f\") " pod="openshift-ingress/e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm"
Sep 09 16:25:50.910438 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:50.910415    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5749d6e6-548c-40af-928a-7f8471c7492f-istiod-ca-cert\") pod \"e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm\" (UID: \"5749d6e6-548c-40af-928a-7f8471c7492f\") " pod="openshift-ingress/e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm"
Sep 09 16:25:50.913674 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:50.910867    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5749d6e6-548c-40af-928a-7f8471c7492f-workload-socket\") pod \"e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm\" (UID: \"5749d6e6-548c-40af-928a-7f8471c7492f\") " pod="openshift-ingress/e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm"
Sep 09 16:25:50.914312 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:50.914292    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5749d6e6-548c-40af-928a-7f8471c7492f-istio-podinfo\") pod \"e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm\" (UID: \"5749d6e6-548c-40af-928a-7f8471c7492f\") " pod="openshift-ingress/e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm"
Sep 09 16:25:50.918488 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:50.915412    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5749d6e6-548c-40af-928a-7f8471c7492f-istio-envoy\") pod \"e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm\" (UID: \"5749d6e6-548c-40af-928a-7f8471c7492f\") " pod="openshift-ingress/e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm"
Sep 09 16:25:50.926226 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:50.926167    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5749d6e6-548c-40af-928a-7f8471c7492f-istio-token\") pod \"e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm\" (UID: \"5749d6e6-548c-40af-928a-7f8471c7492f\") " pod="openshift-ingress/e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm"
Sep 09 16:25:50.927758 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:50.927733    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8zb4\" (UniqueName: \"kubernetes.io/projected/5749d6e6-548c-40af-928a-7f8471c7492f-kube-api-access-m8zb4\") pod \"e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm\" (UID: \"5749d6e6-548c-40af-928a-7f8471c7492f\") " pod="openshift-ingress/e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm"
Sep 09 16:25:50.958480 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:50.958433    2567 patch_prober.go:28] interesting pod/e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Sep 09 16:25:50.958631 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:50.958515    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc" podUID="155986b3-9283-4eea-bff1-479cad0f546f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Sep 09 16:25:51.035965 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:51.035866    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48074e07-e45a-456c-bbab-ff849b31c899" path="/var/lib/kubelet/pods/48074e07-e45a-456c-bbab-ff849b31c899/volumes"
Sep 09 16:25:51.091110 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:51.091073    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm"
Sep 09 16:25:51.390774 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:51.389389    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm"]
Sep 09 16:25:51.416471 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:51.416392    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:25:51.416653 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:51.416567    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:25:51.416723 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:51.416657    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:25:51.958790 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:51.958751    2567 patch_prober.go:28] interesting pod/e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Sep 09 16:25:51.959188 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:51.958809    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc" podUID="155986b3-9283-4eea-bff1-479cad0f546f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Sep 09 16:25:52.226060 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:52.225962    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm" event={"ID":"5749d6e6-548c-40af-928a-7f8471c7492f","Type":"ContainerStarted","Data":"d6b748c92b84717b7e1013c94d090151ab7a76373339f7c62cd4f8f26fcc169a"}
Sep 09 16:25:52.226060 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:52.226017    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm" event={"ID":"5749d6e6-548c-40af-928a-7f8471c7492f","Type":"ContainerStarted","Data":"382f72d47fd4e7bc5ad24881c4963cb41bab655a860cce669f5e5fbc3551f367"}
Sep 09 16:25:52.246097 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:52.246034    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm" podStartSLOduration=2.246016726 podStartE2EDuration="2.246016726s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 16:25:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 16:25:52.244298824 +0000 UTC m=+1027.662148826" watchObservedRunningTime="2026-09-09 16:25:52.246016726 +0000 UTC m=+1027.663866725"
Sep 09 16:25:52.958125 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:52.958092    2567 patch_prober.go:28] interesting pod/e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Sep 09 16:25:52.958292 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:52.958145    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc" podUID="155986b3-9283-4eea-bff1-479cad0f546f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Sep 09 16:25:53.091894 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:53.091599    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm"
Sep 09 16:25:53.109255 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:53.109007    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm"
Sep 09 16:25:53.231111 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:53.231022    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm"
Sep 09 16:25:53.232195 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:53.232178    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm"
Sep 09 16:25:53.958278 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:53.958245    2567 patch_prober.go:28] interesting pod/e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Sep 09 16:25:53.958447 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:53.958297    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc" podUID="155986b3-9283-4eea-bff1-479cad0f546f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Sep 09 16:25:54.899189 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:54.899124    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8"
Sep 09 16:25:54.955909 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:54.955878    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cbqrs\" (UniqueName: \"kubernetes.io/projected/cf6e0304-8dce-4764-8942-133cded65ce5-kube-api-access-cbqrs\") pod \"cf6e0304-8dce-4764-8942-133cded65ce5\" (UID: \"cf6e0304-8dce-4764-8942-133cded65ce5\") "
Sep 09 16:25:54.956103 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:54.955929    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf6e0304-8dce-4764-8942-133cded65ce5-istio-envoy\") pod \"cf6e0304-8dce-4764-8942-133cded65ce5\" (UID: \"cf6e0304-8dce-4764-8942-133cded65ce5\") "
Sep 09 16:25:54.956103 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:54.955946    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf6e0304-8dce-4764-8942-133cded65ce5-istio-data\") pod \"cf6e0304-8dce-4764-8942-133cded65ce5\" (UID: \"cf6e0304-8dce-4764-8942-133cded65ce5\") "
Sep 09 16:25:54.956103 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:54.955973    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf6e0304-8dce-4764-8942-133cded65ce5-workload-socket\") pod \"cf6e0304-8dce-4764-8942-133cded65ce5\" (UID: \"cf6e0304-8dce-4764-8942-133cded65ce5\") "
Sep 09 16:25:54.956103 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:54.955995    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf6e0304-8dce-4764-8942-133cded65ce5-istio-token\") pod \"cf6e0304-8dce-4764-8942-133cded65ce5\" (UID: \"cf6e0304-8dce-4764-8942-133cded65ce5\") "
Sep 09 16:25:54.956103 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:54.956013    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf6e0304-8dce-4764-8942-133cded65ce5-credential-socket\") pod \"cf6e0304-8dce-4764-8942-133cded65ce5\" (UID: \"cf6e0304-8dce-4764-8942-133cded65ce5\") "
Sep 09 16:25:54.956103 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:54.956084    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf6e0304-8dce-4764-8942-133cded65ce5-workload-certs\") pod \"cf6e0304-8dce-4764-8942-133cded65ce5\" (UID: \"cf6e0304-8dce-4764-8942-133cded65ce5\") "
Sep 09 16:25:54.956429 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:54.956140    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf6e0304-8dce-4764-8942-133cded65ce5-istio-podinfo\") pod \"cf6e0304-8dce-4764-8942-133cded65ce5\" (UID: \"cf6e0304-8dce-4764-8942-133cded65ce5\") "
Sep 09 16:25:54.956429 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:54.956168    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf6e0304-8dce-4764-8942-133cded65ce5-istiod-ca-cert\") pod \"cf6e0304-8dce-4764-8942-133cded65ce5\" (UID: \"cf6e0304-8dce-4764-8942-133cded65ce5\") "
Sep 09 16:25:54.956429 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:54.956317    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf6e0304-8dce-4764-8942-133cded65ce5-istio-data" (OuterVolumeSpecName: "istio-data") pod "cf6e0304-8dce-4764-8942-133cded65ce5" (UID: "cf6e0304-8dce-4764-8942-133cded65ce5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:25:54.956429 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:54.956338    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf6e0304-8dce-4764-8942-133cded65ce5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "cf6e0304-8dce-4764-8942-133cded65ce5" (UID: "cf6e0304-8dce-4764-8942-133cded65ce5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:25:54.956429 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:54.956332    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf6e0304-8dce-4764-8942-133cded65ce5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "cf6e0304-8dce-4764-8942-133cded65ce5" (UID: "cf6e0304-8dce-4764-8942-133cded65ce5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:25:54.956859 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:54.956436    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf6e0304-8dce-4764-8942-133cded65ce5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "cf6e0304-8dce-4764-8942-133cded65ce5" (UID: "cf6e0304-8dce-4764-8942-133cded65ce5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:25:54.956859 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:54.956629    2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf6e0304-8dce-4764-8942-133cded65ce5-workload-certs\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:25:54.956859 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:54.956651    2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf6e0304-8dce-4764-8942-133cded65ce5-istio-data\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:25:54.956859 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:54.956665    2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf6e0304-8dce-4764-8942-133cded65ce5-workload-socket\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:25:54.956859 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:54.956665    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf6e0304-8dce-4764-8942-133cded65ce5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "cf6e0304-8dce-4764-8942-133cded65ce5" (UID: "cf6e0304-8dce-4764-8942-133cded65ce5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 16:25:54.956859 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:54.956677    2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf6e0304-8dce-4764-8942-133cded65ce5-credential-socket\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:25:54.958571 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:54.958546    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf6e0304-8dce-4764-8942-133cded65ce5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "cf6e0304-8dce-4764-8942-133cded65ce5" (UID: "cf6e0304-8dce-4764-8942-133cded65ce5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:25:54.958752 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:54.958624    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cf6e0304-8dce-4764-8942-133cded65ce5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "cf6e0304-8dce-4764-8942-133cded65ce5" (UID: "cf6e0304-8dce-4764-8942-133cded65ce5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 16:25:54.959013 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:54.958987    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf6e0304-8dce-4764-8942-133cded65ce5-kube-api-access-cbqrs" (OuterVolumeSpecName: "kube-api-access-cbqrs") pod "cf6e0304-8dce-4764-8942-133cded65ce5" (UID: "cf6e0304-8dce-4764-8942-133cded65ce5"). InnerVolumeSpecName "kube-api-access-cbqrs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 16:25:54.959013 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:54.959001    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf6e0304-8dce-4764-8942-133cded65ce5-istio-token" (OuterVolumeSpecName: "istio-token") pod "cf6e0304-8dce-4764-8942-133cded65ce5" (UID: "cf6e0304-8dce-4764-8942-133cded65ce5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 16:25:54.962663 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:54.962642    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc"
Sep 09 16:25:54.963010 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:54.962992    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc"
Sep 09 16:25:54.963814 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:54.963798    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc"
Sep 09 16:25:55.057856 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:55.057822    2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf6e0304-8dce-4764-8942-133cded65ce5-istio-podinfo\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:25:55.057856 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:55.057856    2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf6e0304-8dce-4764-8942-133cded65ce5-istiod-ca-cert\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:25:55.058047 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:55.057870    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cbqrs\" (UniqueName: \"kubernetes.io/projected/cf6e0304-8dce-4764-8942-133cded65ce5-kube-api-access-cbqrs\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:25:55.058047 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:55.057884    2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf6e0304-8dce-4764-8942-133cded65ce5-istio-envoy\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:25:55.058047 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:55.057900    2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf6e0304-8dce-4764-8942-133cded65ce5-istio-token\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:25:55.243951 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:55.243858    2567 generic.go:356] "Generic (PLEG): container finished" podID="cf6e0304-8dce-4764-8942-133cded65ce5" containerID="7dbd236da2d26ef0f61784571b83ed327d0325f7d5efd5707873822fc9b57885" exitCode=0
Sep 09 16:25:55.243951 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:55.243907    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8"
Sep 09 16:25:55.243951 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:55.243938    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8" event={"ID":"cf6e0304-8dce-4764-8942-133cded65ce5","Type":"ContainerDied","Data":"7dbd236da2d26ef0f61784571b83ed327d0325f7d5efd5707873822fc9b57885"}
Sep 09 16:25:55.244252 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:55.244000    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8" event={"ID":"cf6e0304-8dce-4764-8942-133cded65ce5","Type":"ContainerDied","Data":"985de15a0f61f1af7a1051e902cd9fbf80ad9883a054c3cae44a47916d070170"}
Sep 09 16:25:55.244252 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:55.244031    2567 scope.go:117] "RemoveContainer" containerID="7dbd236da2d26ef0f61784571b83ed327d0325f7d5efd5707873822fc9b57885"
Sep 09 16:25:55.254484 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:55.254438    2567 scope.go:117] "RemoveContainer" containerID="1e65dcf563fa00fe0e055ad3f2b01cfa876c28759f9c710efc4036d0b7e3cbaa"
Sep 09 16:25:55.261193 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:55.261172    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8"]
Sep 09 16:25:55.267668 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:55.267650    2567 scope.go:117] "RemoveContainer" containerID="7dbd236da2d26ef0f61784571b83ed327d0325f7d5efd5707873822fc9b57885"
Sep 09 16:25:55.267964 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:25:55.267943    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7dbd236da2d26ef0f61784571b83ed327d0325f7d5efd5707873822fc9b57885\": container with ID starting with 7dbd236da2d26ef0f61784571b83ed327d0325f7d5efd5707873822fc9b57885 not found: ID does not exist" containerID="7dbd236da2d26ef0f61784571b83ed327d0325f7d5efd5707873822fc9b57885"
Sep 09 16:25:55.268030 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:55.267972    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7dbd236da2d26ef0f61784571b83ed327d0325f7d5efd5707873822fc9b57885"} err="failed to get container status \"7dbd236da2d26ef0f61784571b83ed327d0325f7d5efd5707873822fc9b57885\": rpc error: code = NotFound desc = could not find container \"7dbd236da2d26ef0f61784571b83ed327d0325f7d5efd5707873822fc9b57885\": container with ID starting with 7dbd236da2d26ef0f61784571b83ed327d0325f7d5efd5707873822fc9b57885 not found: ID does not exist"
Sep 09 16:25:55.268030 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:55.267989    2567 scope.go:117] "RemoveContainer" containerID="1e65dcf563fa00fe0e055ad3f2b01cfa876c28759f9c710efc4036d0b7e3cbaa"
Sep 09 16:25:55.268245 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:25:55.268230    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e65dcf563fa00fe0e055ad3f2b01cfa876c28759f9c710efc4036d0b7e3cbaa\": container with ID starting with 1e65dcf563fa00fe0e055ad3f2b01cfa876c28759f9c710efc4036d0b7e3cbaa not found: ID does not exist" containerID="1e65dcf563fa00fe0e055ad3f2b01cfa876c28759f9c710efc4036d0b7e3cbaa"
Sep 09 16:25:55.268285 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:55.268250    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e65dcf563fa00fe0e055ad3f2b01cfa876c28759f9c710efc4036d0b7e3cbaa"} err="failed to get container status \"1e65dcf563fa00fe0e055ad3f2b01cfa876c28759f9c710efc4036d0b7e3cbaa\": rpc error: code = NotFound desc = could not find container \"1e65dcf563fa00fe0e055ad3f2b01cfa876c28759f9c710efc4036d0b7e3cbaa\": container with ID starting with 1e65dcf563fa00fe0e055ad3f2b01cfa876c28759f9c710efc4036d0b7e3cbaa not found: ID does not exist"
Sep 09 16:25:55.269214 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:55.269195    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-0d2e38e1-openshift-default-57c85d8ff6-t92s8"]
Sep 09 16:25:57.035591 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:25:57.035556    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf6e0304-8dce-4764-8942-133cded65ce5" path="/var/lib/kubelet/pods/cf6e0304-8dce-4764-8942-133cded65ce5/volumes"
Sep 09 16:26:02.215279 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:02.215235    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:26:12.348784 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:12.348701    2567 generic.go:356] "Generic (PLEG): container finished" podID="5749d6e6-548c-40af-928a-7f8471c7492f" containerID="d6b748c92b84717b7e1013c94d090151ab7a76373339f7c62cd4f8f26fcc169a" exitCode=0
Sep 09 16:26:12.348784 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:12.348749    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm" event={"ID":"5749d6e6-548c-40af-928a-7f8471c7492f","Type":"ContainerDied","Data":"d6b748c92b84717b7e1013c94d090151ab7a76373339f7c62cd4f8f26fcc169a"}
Sep 09 16:26:12.349219 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:12.349186    2567 scope.go:117] "RemoveContainer" containerID="d6b748c92b84717b7e1013c94d090151ab7a76373339f7c62cd4f8f26fcc169a"
Sep 09 16:26:12.350326 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:12.350291    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:26:12.350380 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:12.350369    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:26:12.350416 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:12.350404    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:26:13.355746 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:13.355717    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm" event={"ID":"5749d6e6-548c-40af-928a-7f8471c7492f","Type":"ContainerStarted","Data":"cd8aea4fd8ebb1cd0aa46c1f03197dc48c1f9f487dc7c885b20117f05e9544b5"}
Sep 09 16:26:14.092028 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:14.091985    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm"
Sep 09 16:26:14.092349 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:14.092318    2567 patch_prober.go:28] interesting pod/e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body=
Sep 09 16:26:14.092443 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:14.092381    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm" podUID="5749d6e6-548c-40af-928a-7f8471c7492f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused"
Sep 09 16:26:15.092259 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:15.092211    2567 patch_prober.go:28] interesting pod/e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body=
Sep 09 16:26:15.092785 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:15.092281    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm" podUID="5749d6e6-548c-40af-928a-7f8471c7492f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused"
Sep 09 16:26:16.091952 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:16.091912    2567 patch_prober.go:28] interesting pod/e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body=
Sep 09 16:26:16.092233 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:16.092206    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm" podUID="5749d6e6-548c-40af-928a-7f8471c7492f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused"
Sep 09 16:26:17.091599 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:17.091558    2567 patch_prober.go:28] interesting pod/e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body=
Sep 09 16:26:17.092208 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:17.092170    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm" podUID="5749d6e6-548c-40af-928a-7f8471c7492f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused"
Sep 09 16:26:18.092106 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:18.092066    2567 patch_prober.go:28] interesting pod/e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body=
Sep 09 16:26:18.092656 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:18.092178    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm" podUID="5749d6e6-548c-40af-928a-7f8471c7492f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused"
Sep 09 16:26:18.523778 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:18.523689    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:26:18.878681 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:18.878643    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fb79f974-6jjhs"]
Sep 09 16:26:18.879057 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:18.879043    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf6e0304-8dce-4764-8942-133cded65ce5" containerName="istio-proxy"
Sep 09 16:26:18.879113 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:18.879059    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf6e0304-8dce-4764-8942-133cded65ce5" containerName="istio-proxy"
Sep 09 16:26:18.879113 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:18.879072    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf6e0304-8dce-4764-8942-133cded65ce5" containerName="istio-proxy"
Sep 09 16:26:18.879113 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:18.879080    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf6e0304-8dce-4764-8942-133cded65ce5" containerName="istio-proxy"
Sep 09 16:26:18.879217 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:18.879151    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf6e0304-8dce-4764-8942-133cded65ce5" containerName="istio-proxy"
Sep 09 16:26:18.879217 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:18.879160    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf6e0304-8dce-4764-8942-133cded65ce5" containerName="istio-proxy"
Sep 09 16:26:18.882370 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:18.882319    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fb79f974-6jjhs"
Sep 09 16:26:18.884692 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:18.884669    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 09 16:26:18.884826 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:18.884741    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wxghs\""
Sep 09 16:26:18.889286 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:18.889257    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fb79f974-6jjhs"]
Sep 09 16:26:18.990432 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:18.990384    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0295021c-2adf-4399-bb64-8ee98b15da42-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-fb79f974-6jjhs\" (UID: \"0295021c-2adf-4399-bb64-8ee98b15da42\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fb79f974-6jjhs"
Sep 09 16:26:19.092155 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:19.092097    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0295021c-2adf-4399-bb64-8ee98b15da42-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-fb79f974-6jjhs\" (UID: \"0295021c-2adf-4399-bb64-8ee98b15da42\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fb79f974-6jjhs"
Sep 09 16:26:19.093015 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:19.092991    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0295021c-2adf-4399-bb64-8ee98b15da42-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-fb79f974-6jjhs\" (UID: \"0295021c-2adf-4399-bb64-8ee98b15da42\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fb79f974-6jjhs"
Sep 09 16:26:19.110853 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:19.110816    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm"
Sep 09 16:26:19.111121 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:19.111099    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm"
Sep 09 16:26:19.111931 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:19.111910    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm"
Sep 09 16:26:19.196898 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:19.196577    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fb79f974-6jjhs"
Sep 09 16:26:19.473229 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:19.473204    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fb79f974-6jjhs"]
Sep 09 16:26:19.475817 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:26:19.475785    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0295021c_2adf_4399_bb64_8ee98b15da42.slice/crio-9dbdcbde0d5c861905d17e4d5e5370f7eee02923b53ca1e6a6f17ece41b82b99 WatchSource:0}: Error finding container 9dbdcbde0d5c861905d17e4d5e5370f7eee02923b53ca1e6a6f17ece41b82b99: Status 404 returned error can't find the container with id 9dbdcbde0d5c861905d17e4d5e5370f7eee02923b53ca1e6a6f17ece41b82b99
Sep 09 16:26:20.390778 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:20.390744    2567 generic.go:356] "Generic (PLEG): container finished" podID="0295021c-2adf-4399-bb64-8ee98b15da42" containerID="ded8c57af835b9b33dd8d85e91593d2d6e1316182c7a7137e62149a85aa0223c" exitCode=0
Sep 09 16:26:20.391141 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:20.390831    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fb79f974-6jjhs" event={"ID":"0295021c-2adf-4399-bb64-8ee98b15da42","Type":"ContainerDied","Data":"ded8c57af835b9b33dd8d85e91593d2d6e1316182c7a7137e62149a85aa0223c"}
Sep 09 16:26:20.391141 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:20.390877    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fb79f974-6jjhs" event={"ID":"0295021c-2adf-4399-bb64-8ee98b15da42","Type":"ContainerStarted","Data":"9dbdcbde0d5c861905d17e4d5e5370f7eee02923b53ca1e6a6f17ece41b82b99"}
Sep 09 16:26:21.550801 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:21.550774    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fb79f974-6jjhs"
Sep 09 16:26:21.722107 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:21.722017    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0295021c-2adf-4399-bb64-8ee98b15da42-maas-api-service-ca\") pod \"0295021c-2adf-4399-bb64-8ee98b15da42\" (UID: \"0295021c-2adf-4399-bb64-8ee98b15da42\") "
Sep 09 16:26:21.722361 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:21.722338    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0295021c-2adf-4399-bb64-8ee98b15da42-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0295021c-2adf-4399-bb64-8ee98b15da42" (UID: "0295021c-2adf-4399-bb64-8ee98b15da42"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 16:26:21.823486 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:21.823425    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0295021c-2adf-4399-bb64-8ee98b15da42-maas-api-service-ca\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:26:22.404362 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:22.404327    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fb79f974-6jjhs"
Sep 09 16:26:22.404362 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:22.404361    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fb79f974-6jjhs" event={"ID":"0295021c-2adf-4399-bb64-8ee98b15da42","Type":"ContainerDied","Data":"9dbdcbde0d5c861905d17e4d5e5370f7eee02923b53ca1e6a6f17ece41b82b99"}
Sep 09 16:26:22.404588 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:22.404401    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9dbdcbde0d5c861905d17e4d5e5370f7eee02923b53ca1e6a6f17ece41b82b99"
Sep 09 16:26:25.525380 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:25.525339    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fb79f974-6jjhs"]
Sep 09 16:26:25.526996 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:25.526977    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fb79f974-6jjhs"]
Sep 09 16:26:25.685885 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:25.685835    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm"]
Sep 09 16:26:25.686431 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:25.686363    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm" podUID="5749d6e6-548c-40af-928a-7f8471c7492f" containerName="istio-proxy" containerID="cri-o://cd8aea4fd8ebb1cd0aa46c1f03197dc48c1f9f487dc7c885b20117f05e9544b5" gracePeriod=30
Sep 09 16:26:26.590511 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:26.590476    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh"]
Sep 09 16:26:26.590914 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:26.590897    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0295021c-2adf-4399-bb64-8ee98b15da42" containerName="revoke-keys"
Sep 09 16:26:26.590963 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:26.590915    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="0295021c-2adf-4399-bb64-8ee98b15da42" containerName="revoke-keys"
Sep 09 16:26:26.590998 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:26.590980    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="0295021c-2adf-4399-bb64-8ee98b15da42" containerName="revoke-keys"
Sep 09 16:26:26.594275 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:26.594256    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh"
Sep 09 16:26:26.596806 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:26.596777    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-8a375fa0-openshift-default-dockercfg-ddv87\""
Sep 09 16:26:26.605848 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:26.605821    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh"]
Sep 09 16:26:26.778143 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:26.778108    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1902f142-3560-4d43-94d0-f8041d1eb0b3-istiod-ca-cert\") pod \"e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh\" (UID: \"1902f142-3560-4d43-94d0-f8041d1eb0b3\") " pod="openshift-ingress/e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh"
Sep 09 16:26:26.778325 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:26.778155    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1902f142-3560-4d43-94d0-f8041d1eb0b3-credential-socket\") pod \"e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh\" (UID: \"1902f142-3560-4d43-94d0-f8041d1eb0b3\") " pod="openshift-ingress/e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh"
Sep 09 16:26:26.778325 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:26.778179    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1902f142-3560-4d43-94d0-f8041d1eb0b3-workload-certs\") pod \"e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh\" (UID: \"1902f142-3560-4d43-94d0-f8041d1eb0b3\") " pod="openshift-ingress/e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh"
Sep 09 16:26:26.778325 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:26.778205    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1902f142-3560-4d43-94d0-f8041d1eb0b3-istio-token\") pod \"e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh\" (UID: \"1902f142-3560-4d43-94d0-f8041d1eb0b3\") " pod="openshift-ingress/e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh"
Sep 09 16:26:26.778325 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:26.778255    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1902f142-3560-4d43-94d0-f8041d1eb0b3-istio-data\") pod \"e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh\" (UID: \"1902f142-3560-4d43-94d0-f8041d1eb0b3\") " pod="openshift-ingress/e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh"
Sep 09 16:26:26.778325 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:26.778312    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmht7\" (UniqueName: \"kubernetes.io/projected/1902f142-3560-4d43-94d0-f8041d1eb0b3-kube-api-access-jmht7\") pod \"e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh\" (UID: \"1902f142-3560-4d43-94d0-f8041d1eb0b3\") " pod="openshift-ingress/e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh"
Sep 09 16:26:26.778587 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:26.778364    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1902f142-3560-4d43-94d0-f8041d1eb0b3-istio-envoy\") pod \"e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh\" (UID: \"1902f142-3560-4d43-94d0-f8041d1eb0b3\") " pod="openshift-ingress/e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh"
Sep 09 16:26:26.778587 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:26.778386    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1902f142-3560-4d43-94d0-f8041d1eb0b3-workload-socket\") pod \"e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh\" (UID: \"1902f142-3560-4d43-94d0-f8041d1eb0b3\") " pod="openshift-ingress/e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh"
Sep 09 16:26:26.778587 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:26.778425    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1902f142-3560-4d43-94d0-f8041d1eb0b3-istio-podinfo\") pod \"e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh\" (UID: \"1902f142-3560-4d43-94d0-f8041d1eb0b3\") " pod="openshift-ingress/e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh"
Sep 09 16:26:26.879706 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:26.879613    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1902f142-3560-4d43-94d0-f8041d1eb0b3-workload-certs\") pod \"e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh\" (UID: \"1902f142-3560-4d43-94d0-f8041d1eb0b3\") " pod="openshift-ingress/e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh"
Sep 09 16:26:26.879706 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:26.879651    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1902f142-3560-4d43-94d0-f8041d1eb0b3-istio-token\") pod \"e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh\" (UID: \"1902f142-3560-4d43-94d0-f8041d1eb0b3\") " pod="openshift-ingress/e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh"
Sep 09 16:26:26.879941 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:26.879710    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1902f142-3560-4d43-94d0-f8041d1eb0b3-istio-data\") pod \"e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh\" (UID: \"1902f142-3560-4d43-94d0-f8041d1eb0b3\") " pod="openshift-ingress/e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh"
Sep 09 16:26:26.879941 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:26.879744    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmht7\" (UniqueName: \"kubernetes.io/projected/1902f142-3560-4d43-94d0-f8041d1eb0b3-kube-api-access-jmht7\") pod \"e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh\" (UID: \"1902f142-3560-4d43-94d0-f8041d1eb0b3\") " pod="openshift-ingress/e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh"
Sep 09 16:26:26.879941 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:26.879800    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1902f142-3560-4d43-94d0-f8041d1eb0b3-istio-envoy\") pod \"e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh\" (UID: \"1902f142-3560-4d43-94d0-f8041d1eb0b3\") " pod="openshift-ingress/e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh"
Sep 09 16:26:26.879941 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:26.879826    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1902f142-3560-4d43-94d0-f8041d1eb0b3-workload-socket\") pod \"e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh\" (UID: \"1902f142-3560-4d43-94d0-f8041d1eb0b3\") " pod="openshift-ingress/e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh"
Sep 09 16:26:26.879941 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:26.879865    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1902f142-3560-4d43-94d0-f8041d1eb0b3-istio-podinfo\") pod \"e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh\" (UID: \"1902f142-3560-4d43-94d0-f8041d1eb0b3\") " pod="openshift-ingress/e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh"
Sep 09 16:26:26.879941 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:26.879910    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1902f142-3560-4d43-94d0-f8041d1eb0b3-istiod-ca-cert\") pod \"e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh\" (UID: \"1902f142-3560-4d43-94d0-f8041d1eb0b3\") " pod="openshift-ingress/e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh"
Sep 09 16:26:26.880225 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:26.879956    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1902f142-3560-4d43-94d0-f8041d1eb0b3-credential-socket\") pod \"e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh\" (UID: \"1902f142-3560-4d43-94d0-f8041d1eb0b3\") " pod="openshift-ingress/e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh"
Sep 09 16:26:26.880225 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:26.880054    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1902f142-3560-4d43-94d0-f8041d1eb0b3-workload-certs\") pod \"e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh\" (UID: \"1902f142-3560-4d43-94d0-f8041d1eb0b3\") " pod="openshift-ingress/e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh"
Sep 09 16:26:26.880225 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:26.880092    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1902f142-3560-4d43-94d0-f8041d1eb0b3-istio-data\") pod \"e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh\" (UID: \"1902f142-3560-4d43-94d0-f8041d1eb0b3\") " pod="openshift-ingress/e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh"
Sep 09 16:26:26.880391 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:26.880274    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1902f142-3560-4d43-94d0-f8041d1eb0b3-workload-socket\") pod \"e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh\" (UID: \"1902f142-3560-4d43-94d0-f8041d1eb0b3\") " pod="openshift-ingress/e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh"
Sep 09 16:26:26.880391 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:26.880371    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1902f142-3560-4d43-94d0-f8041d1eb0b3-credential-socket\") pod \"e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh\" (UID: \"1902f142-3560-4d43-94d0-f8041d1eb0b3\") " pod="openshift-ingress/e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh"
Sep 09 16:26:26.880652 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:26.880636    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1902f142-3560-4d43-94d0-f8041d1eb0b3-istiod-ca-cert\") pod \"e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh\" (UID: \"1902f142-3560-4d43-94d0-f8041d1eb0b3\") " pod="openshift-ingress/e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh"
Sep 09 16:26:26.882149 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:26.882126    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1902f142-3560-4d43-94d0-f8041d1eb0b3-istio-envoy\") pod \"e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh\" (UID: \"1902f142-3560-4d43-94d0-f8041d1eb0b3\") " pod="openshift-ingress/e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh"
Sep 09 16:26:26.882586 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:26.882564    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1902f142-3560-4d43-94d0-f8041d1eb0b3-istio-podinfo\") pod \"e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh\" (UID: \"1902f142-3560-4d43-94d0-f8041d1eb0b3\") " pod="openshift-ingress/e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh"
Sep 09 16:26:26.887666 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:26.887638    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1902f142-3560-4d43-94d0-f8041d1eb0b3-istio-token\") pod \"e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh\" (UID: \"1902f142-3560-4d43-94d0-f8041d1eb0b3\") " pod="openshift-ingress/e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh"
Sep 09 16:26:26.887885 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:26.887866    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmht7\" (UniqueName: \"kubernetes.io/projected/1902f142-3560-4d43-94d0-f8041d1eb0b3-kube-api-access-jmht7\") pod \"e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh\" (UID: \"1902f142-3560-4d43-94d0-f8041d1eb0b3\") " pod="openshift-ingress/e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh"
Sep 09 16:26:26.905833 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:26.905810    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh"
Sep 09 16:26:27.036319 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:27.036287    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0295021c-2adf-4399-bb64-8ee98b15da42" path="/var/lib/kubelet/pods/0295021c-2adf-4399-bb64-8ee98b15da42/volumes"
Sep 09 16:26:27.181235 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:27.181211    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh"]
Sep 09 16:26:27.183222 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:26:27.183199    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1902f142_3560_4d43_94d0_f8041d1eb0b3.slice/crio-9c191de3d3a615504f99694ea5c28f3e784c1f2a9f98ee6f0b56b29214e6a058 WatchSource:0}: Error finding container 9c191de3d3a615504f99694ea5c28f3e784c1f2a9f98ee6f0b56b29214e6a058: Status 404 returned error can't find the container with id 9c191de3d3a615504f99694ea5c28f3e784c1f2a9f98ee6f0b56b29214e6a058
Sep 09 16:26:27.185580 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:27.185548    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:26:27.185652 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:27.185613    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:26:27.185652 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:27.185644    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:26:27.431308 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:27.431212    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh" event={"ID":"1902f142-3560-4d43-94d0-f8041d1eb0b3","Type":"ContainerStarted","Data":"a682f7bf7af555d9f16dbee55b829aef329a4392e085eeddc384357054fa427d"}
Sep 09 16:26:27.431308 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:27.431277    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh" event={"ID":"1902f142-3560-4d43-94d0-f8041d1eb0b3","Type":"ContainerStarted","Data":"9c191de3d3a615504f99694ea5c28f3e784c1f2a9f98ee6f0b56b29214e6a058"}
Sep 09 16:26:27.451243 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:27.451181    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh" podStartSLOduration=1.451155569 podStartE2EDuration="1.451155569s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 16:26:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 16:26:27.450412139 +0000 UTC m=+1062.868262139" watchObservedRunningTime="2026-09-09 16:26:27.451155569 +0000 UTC m=+1062.869005570"
Sep 09 16:26:27.906311 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:27.906276    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh"
Sep 09 16:26:28.911194 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:28.911162    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh"
Sep 09 16:26:29.441146 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:29.441113    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh"
Sep 09 16:26:29.442201 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:29.442182    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh"
Sep 09 16:26:30.961426 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:30.961357    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm"
Sep 09 16:26:31.020014 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:31.019979    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5749d6e6-548c-40af-928a-7f8471c7492f-workload-certs\") pod \"5749d6e6-548c-40af-928a-7f8471c7492f\" (UID: \"5749d6e6-548c-40af-928a-7f8471c7492f\") "
Sep 09 16:26:31.020208 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:31.020023    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5749d6e6-548c-40af-928a-7f8471c7492f-istio-envoy\") pod \"5749d6e6-548c-40af-928a-7f8471c7492f\" (UID: \"5749d6e6-548c-40af-928a-7f8471c7492f\") "
Sep 09 16:26:31.020208 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:31.020067    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m8zb4\" (UniqueName: \"kubernetes.io/projected/5749d6e6-548c-40af-928a-7f8471c7492f-kube-api-access-m8zb4\") pod \"5749d6e6-548c-40af-928a-7f8471c7492f\" (UID: \"5749d6e6-548c-40af-928a-7f8471c7492f\") "
Sep 09 16:26:31.020208 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:31.020103    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5749d6e6-548c-40af-928a-7f8471c7492f-workload-socket\") pod \"5749d6e6-548c-40af-928a-7f8471c7492f\" (UID: \"5749d6e6-548c-40af-928a-7f8471c7492f\") "
Sep 09 16:26:31.020208 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:31.020147    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5749d6e6-548c-40af-928a-7f8471c7492f-istio-podinfo\") pod \"5749d6e6-548c-40af-928a-7f8471c7492f\" (UID: \"5749d6e6-548c-40af-928a-7f8471c7492f\") "
Sep 09 16:26:31.020208 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:31.020183    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5749d6e6-548c-40af-928a-7f8471c7492f-istio-token\") pod \"5749d6e6-548c-40af-928a-7f8471c7492f\" (UID: \"5749d6e6-548c-40af-928a-7f8471c7492f\") "
Sep 09 16:26:31.020650 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:31.020236    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5749d6e6-548c-40af-928a-7f8471c7492f-istio-data\") pod \"5749d6e6-548c-40af-928a-7f8471c7492f\" (UID: \"5749d6e6-548c-40af-928a-7f8471c7492f\") "
Sep 09 16:26:31.020650 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:31.020267    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5749d6e6-548c-40af-928a-7f8471c7492f-istiod-ca-cert\") pod \"5749d6e6-548c-40af-928a-7f8471c7492f\" (UID: \"5749d6e6-548c-40af-928a-7f8471c7492f\") "
Sep 09 16:26:31.020650 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:31.020308    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5749d6e6-548c-40af-928a-7f8471c7492f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5749d6e6-548c-40af-928a-7f8471c7492f" (UID: "5749d6e6-548c-40af-928a-7f8471c7492f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:26:31.020650 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:31.020323    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5749d6e6-548c-40af-928a-7f8471c7492f-credential-socket\") pod \"5749d6e6-548c-40af-928a-7f8471c7492f\" (UID: \"5749d6e6-548c-40af-928a-7f8471c7492f\") "
Sep 09 16:26:31.020650 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:31.020483    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5749d6e6-548c-40af-928a-7f8471c7492f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5749d6e6-548c-40af-928a-7f8471c7492f" (UID: "5749d6e6-548c-40af-928a-7f8471c7492f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:26:31.020913 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:31.020793    2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5749d6e6-548c-40af-928a-7f8471c7492f-workload-socket\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:26:31.020913 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:31.020815    2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5749d6e6-548c-40af-928a-7f8471c7492f-workload-certs\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:26:31.020913 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:31.020819    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5749d6e6-548c-40af-928a-7f8471c7492f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5749d6e6-548c-40af-928a-7f8471c7492f" (UID: "5749d6e6-548c-40af-928a-7f8471c7492f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 16:26:31.021063 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:31.021034    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5749d6e6-548c-40af-928a-7f8471c7492f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5749d6e6-548c-40af-928a-7f8471c7492f" (UID: "5749d6e6-548c-40af-928a-7f8471c7492f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:26:31.021136 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:31.021098    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5749d6e6-548c-40af-928a-7f8471c7492f-istio-data" (OuterVolumeSpecName: "istio-data") pod "5749d6e6-548c-40af-928a-7f8471c7492f" (UID: "5749d6e6-548c-40af-928a-7f8471c7492f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:26:31.022671 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:31.022646    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5749d6e6-548c-40af-928a-7f8471c7492f-kube-api-access-m8zb4" (OuterVolumeSpecName: "kube-api-access-m8zb4") pod "5749d6e6-548c-40af-928a-7f8471c7492f" (UID: "5749d6e6-548c-40af-928a-7f8471c7492f"). InnerVolumeSpecName "kube-api-access-m8zb4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 16:26:31.022784 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:31.022679    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5749d6e6-548c-40af-928a-7f8471c7492f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5749d6e6-548c-40af-928a-7f8471c7492f" (UID: "5749d6e6-548c-40af-928a-7f8471c7492f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:26:31.022784 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:31.022703    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5749d6e6-548c-40af-928a-7f8471c7492f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5749d6e6-548c-40af-928a-7f8471c7492f" (UID: "5749d6e6-548c-40af-928a-7f8471c7492f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 16:26:31.022966 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:31.022947    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5749d6e6-548c-40af-928a-7f8471c7492f-istio-token" (OuterVolumeSpecName: "istio-token") pod "5749d6e6-548c-40af-928a-7f8471c7492f" (UID: "5749d6e6-548c-40af-928a-7f8471c7492f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 16:26:31.121337 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:31.121301    2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5749d6e6-548c-40af-928a-7f8471c7492f-istiod-ca-cert\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:26:31.121337 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:31.121328    2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5749d6e6-548c-40af-928a-7f8471c7492f-credential-socket\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:26:31.121337 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:31.121338    2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5749d6e6-548c-40af-928a-7f8471c7492f-istio-envoy\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:26:31.121337 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:31.121347    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m8zb4\" (UniqueName: \"kubernetes.io/projected/5749d6e6-548c-40af-928a-7f8471c7492f-kube-api-access-m8zb4\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:26:31.121646 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:31.121356    2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5749d6e6-548c-40af-928a-7f8471c7492f-istio-podinfo\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:26:31.121646 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:31.121364    2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5749d6e6-548c-40af-928a-7f8471c7492f-istio-token\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:26:31.121646 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:31.121372    2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5749d6e6-548c-40af-928a-7f8471c7492f-istio-data\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:26:31.453565 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:31.453522    2567 generic.go:356] "Generic (PLEG): container finished" podID="5749d6e6-548c-40af-928a-7f8471c7492f" containerID="cd8aea4fd8ebb1cd0aa46c1f03197dc48c1f9f487dc7c885b20117f05e9544b5" exitCode=0
Sep 09 16:26:31.453565 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:31.453561    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm" event={"ID":"5749d6e6-548c-40af-928a-7f8471c7492f","Type":"ContainerDied","Data":"cd8aea4fd8ebb1cd0aa46c1f03197dc48c1f9f487dc7c885b20117f05e9544b5"}
Sep 09 16:26:31.453825 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:31.453593    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm"
Sep 09 16:26:31.453825 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:31.453629    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm" event={"ID":"5749d6e6-548c-40af-928a-7f8471c7492f","Type":"ContainerDied","Data":"382f72d47fd4e7bc5ad24881c4963cb41bab655a860cce669f5e5fbc3551f367"}
Sep 09 16:26:31.453825 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:31.453659    2567 scope.go:117] "RemoveContainer" containerID="cd8aea4fd8ebb1cd0aa46c1f03197dc48c1f9f487dc7c885b20117f05e9544b5"
Sep 09 16:26:31.462691 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:31.462676    2567 scope.go:117] "RemoveContainer" containerID="d6b748c92b84717b7e1013c94d090151ab7a76373339f7c62cd4f8f26fcc169a"
Sep 09 16:26:31.470750 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:31.470724    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm"]
Sep 09 16:26:31.471234 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:31.471218    2567 scope.go:117] "RemoveContainer" containerID="cd8aea4fd8ebb1cd0aa46c1f03197dc48c1f9f487dc7c885b20117f05e9544b5"
Sep 09 16:26:31.471531 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:26:31.471514    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd8aea4fd8ebb1cd0aa46c1f03197dc48c1f9f487dc7c885b20117f05e9544b5\": container with ID starting with cd8aea4fd8ebb1cd0aa46c1f03197dc48c1f9f487dc7c885b20117f05e9544b5 not found: ID does not exist" containerID="cd8aea4fd8ebb1cd0aa46c1f03197dc48c1f9f487dc7c885b20117f05e9544b5"
Sep 09 16:26:31.471581 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:31.471540    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd8aea4fd8ebb1cd0aa46c1f03197dc48c1f9f487dc7c885b20117f05e9544b5"} err="failed to get container status \"cd8aea4fd8ebb1cd0aa46c1f03197dc48c1f9f487dc7c885b20117f05e9544b5\": rpc error: code = NotFound desc = could not find container \"cd8aea4fd8ebb1cd0aa46c1f03197dc48c1f9f487dc7c885b20117f05e9544b5\": container with ID starting with cd8aea4fd8ebb1cd0aa46c1f03197dc48c1f9f487dc7c885b20117f05e9544b5 not found: ID does not exist"
Sep 09 16:26:31.471581 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:31.471557    2567 scope.go:117] "RemoveContainer" containerID="d6b748c92b84717b7e1013c94d090151ab7a76373339f7c62cd4f8f26fcc169a"
Sep 09 16:26:31.471823 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:26:31.471803    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6b748c92b84717b7e1013c94d090151ab7a76373339f7c62cd4f8f26fcc169a\": container with ID starting with d6b748c92b84717b7e1013c94d090151ab7a76373339f7c62cd4f8f26fcc169a not found: ID does not exist" containerID="d6b748c92b84717b7e1013c94d090151ab7a76373339f7c62cd4f8f26fcc169a"
Sep 09 16:26:31.471881 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:31.471827    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6b748c92b84717b7e1013c94d090151ab7a76373339f7c62cd4f8f26fcc169a"} err="failed to get container status \"d6b748c92b84717b7e1013c94d090151ab7a76373339f7c62cd4f8f26fcc169a\": rpc error: code = NotFound desc = could not find container \"d6b748c92b84717b7e1013c94d090151ab7a76373339f7c62cd4f8f26fcc169a\": container with ID starting with d6b748c92b84717b7e1013c94d090151ab7a76373339f7c62cd4f8f26fcc169a not found: ID does not exist"
Sep 09 16:26:31.474740 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:31.474718    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-fb79f974-gw-openshift-default-6fdd9bc9d8-px9cm"]
Sep 09 16:26:33.036808 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:33.036776    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5749d6e6-548c-40af-928a-7f8471c7492f" path="/var/lib/kubelet/pods/5749d6e6-548c-40af-928a-7f8471c7492f/volumes"
Sep 09 16:26:34.611123 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:34.611090    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:26:43.304971 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:43.304932    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-6d44b6/test-model-6d44b6-kserve-67d6d975d9-xw82r"]
Sep 09 16:26:43.305357 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:43.305314    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5749d6e6-548c-40af-928a-7f8471c7492f" containerName="istio-proxy"
Sep 09 16:26:43.305357 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:43.305325    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="5749d6e6-548c-40af-928a-7f8471c7492f" containerName="istio-proxy"
Sep 09 16:26:43.305437 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:43.305390    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="5749d6e6-548c-40af-928a-7f8471c7492f" containerName="istio-proxy"
Sep 09 16:26:43.308487 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:43.308466    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-6d44b6/test-model-6d44b6-kserve-67d6d975d9-xw82r"
Sep 09 16:26:43.311705 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:43.311662    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-6d44b6\"/\"default-dockercfg-hcqvz\""
Sep 09 16:26:43.311705 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:43.311691    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-6d44b6\"/\"openshift-service-ca.crt\""
Sep 09 16:26:43.311933 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:43.311662    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-6d44b6\"/\"test-model-6d44b6-kserve-self-signed-certs\""
Sep 09 16:26:43.311933 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:43.311765    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-6d44b6\"/\"kube-root-ca.crt\""
Sep 09 16:26:43.318236 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:43.318211    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-6d44b6/test-model-6d44b6-kserve-67d6d975d9-xw82r"]
Sep 09 16:26:43.451751 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:43.451712    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd73e10d-18bb-42fd-be79-ae6949f19f36-dshm\") pod \"test-model-6d44b6-kserve-67d6d975d9-xw82r\" (UID: \"dd73e10d-18bb-42fd-be79-ae6949f19f36\") " pod="ai-tenant-e2e-inf-a-6d44b6/test-model-6d44b6-kserve-67d6d975d9-xw82r"
Sep 09 16:26:43.451751 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:43.451752    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd73e10d-18bb-42fd-be79-ae6949f19f36-tls-certs\") pod \"test-model-6d44b6-kserve-67d6d975d9-xw82r\" (UID: \"dd73e10d-18bb-42fd-be79-ae6949f19f36\") " pod="ai-tenant-e2e-inf-a-6d44b6/test-model-6d44b6-kserve-67d6d975d9-xw82r"
Sep 09 16:26:43.452017 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:43.451777    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgmfb\" (UniqueName: \"kubernetes.io/projected/dd73e10d-18bb-42fd-be79-ae6949f19f36-kube-api-access-xgmfb\") pod \"test-model-6d44b6-kserve-67d6d975d9-xw82r\" (UID: \"dd73e10d-18bb-42fd-be79-ae6949f19f36\") " pod="ai-tenant-e2e-inf-a-6d44b6/test-model-6d44b6-kserve-67d6d975d9-xw82r"
Sep 09 16:26:43.452017 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:43.451849    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd73e10d-18bb-42fd-be79-ae6949f19f36-tmp-dir\") pod \"test-model-6d44b6-kserve-67d6d975d9-xw82r\" (UID: \"dd73e10d-18bb-42fd-be79-ae6949f19f36\") " pod="ai-tenant-e2e-inf-a-6d44b6/test-model-6d44b6-kserve-67d6d975d9-xw82r"
Sep 09 16:26:43.452017 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:43.451917    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd73e10d-18bb-42fd-be79-ae6949f19f36-home\") pod \"test-model-6d44b6-kserve-67d6d975d9-xw82r\" (UID: \"dd73e10d-18bb-42fd-be79-ae6949f19f36\") " pod="ai-tenant-e2e-inf-a-6d44b6/test-model-6d44b6-kserve-67d6d975d9-xw82r"
Sep 09 16:26:43.452017 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:43.452008    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd73e10d-18bb-42fd-be79-ae6949f19f36-model-cache\") pod \"test-model-6d44b6-kserve-67d6d975d9-xw82r\" (UID: \"dd73e10d-18bb-42fd-be79-ae6949f19f36\") " pod="ai-tenant-e2e-inf-a-6d44b6/test-model-6d44b6-kserve-67d6d975d9-xw82r"
Sep 09 16:26:43.553572 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:43.553530    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd73e10d-18bb-42fd-be79-ae6949f19f36-tmp-dir\") pod \"test-model-6d44b6-kserve-67d6d975d9-xw82r\" (UID: \"dd73e10d-18bb-42fd-be79-ae6949f19f36\") " pod="ai-tenant-e2e-inf-a-6d44b6/test-model-6d44b6-kserve-67d6d975d9-xw82r"
Sep 09 16:26:43.553764 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:43.553612    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd73e10d-18bb-42fd-be79-ae6949f19f36-home\") pod \"test-model-6d44b6-kserve-67d6d975d9-xw82r\" (UID: \"dd73e10d-18bb-42fd-be79-ae6949f19f36\") " pod="ai-tenant-e2e-inf-a-6d44b6/test-model-6d44b6-kserve-67d6d975d9-xw82r"
Sep 09 16:26:43.553764 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:43.553655    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd73e10d-18bb-42fd-be79-ae6949f19f36-model-cache\") pod \"test-model-6d44b6-kserve-67d6d975d9-xw82r\" (UID: \"dd73e10d-18bb-42fd-be79-ae6949f19f36\") " pod="ai-tenant-e2e-inf-a-6d44b6/test-model-6d44b6-kserve-67d6d975d9-xw82r"
Sep 09 16:26:43.553764 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:43.553728    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd73e10d-18bb-42fd-be79-ae6949f19f36-dshm\") pod \"test-model-6d44b6-kserve-67d6d975d9-xw82r\" (UID: \"dd73e10d-18bb-42fd-be79-ae6949f19f36\") " pod="ai-tenant-e2e-inf-a-6d44b6/test-model-6d44b6-kserve-67d6d975d9-xw82r"
Sep 09 16:26:43.553764 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:43.553744    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd73e10d-18bb-42fd-be79-ae6949f19f36-tls-certs\") pod \"test-model-6d44b6-kserve-67d6d975d9-xw82r\" (UID: \"dd73e10d-18bb-42fd-be79-ae6949f19f36\") " pod="ai-tenant-e2e-inf-a-6d44b6/test-model-6d44b6-kserve-67d6d975d9-xw82r"
Sep 09 16:26:43.553983 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:43.553764    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgmfb\" (UniqueName: \"kubernetes.io/projected/dd73e10d-18bb-42fd-be79-ae6949f19f36-kube-api-access-xgmfb\") pod \"test-model-6d44b6-kserve-67d6d975d9-xw82r\" (UID: \"dd73e10d-18bb-42fd-be79-ae6949f19f36\") " pod="ai-tenant-e2e-inf-a-6d44b6/test-model-6d44b6-kserve-67d6d975d9-xw82r"
Sep 09 16:26:43.554039 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:43.554003    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd73e10d-18bb-42fd-be79-ae6949f19f36-tmp-dir\") pod \"test-model-6d44b6-kserve-67d6d975d9-xw82r\" (UID: \"dd73e10d-18bb-42fd-be79-ae6949f19f36\") " pod="ai-tenant-e2e-inf-a-6d44b6/test-model-6d44b6-kserve-67d6d975d9-xw82r"
Sep 09 16:26:43.554257 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:43.554232    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd73e10d-18bb-42fd-be79-ae6949f19f36-model-cache\") pod \"test-model-6d44b6-kserve-67d6d975d9-xw82r\" (UID: \"dd73e10d-18bb-42fd-be79-ae6949f19f36\") " pod="ai-tenant-e2e-inf-a-6d44b6/test-model-6d44b6-kserve-67d6d975d9-xw82r"
Sep 09 16:26:43.554399 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:43.554293    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd73e10d-18bb-42fd-be79-ae6949f19f36-home\") pod \"test-model-6d44b6-kserve-67d6d975d9-xw82r\" (UID: \"dd73e10d-18bb-42fd-be79-ae6949f19f36\") " pod="ai-tenant-e2e-inf-a-6d44b6/test-model-6d44b6-kserve-67d6d975d9-xw82r"
Sep 09 16:26:43.556612 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:43.556537    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd73e10d-18bb-42fd-be79-ae6949f19f36-dshm\") pod \"test-model-6d44b6-kserve-67d6d975d9-xw82r\" (UID: \"dd73e10d-18bb-42fd-be79-ae6949f19f36\") " pod="ai-tenant-e2e-inf-a-6d44b6/test-model-6d44b6-kserve-67d6d975d9-xw82r"
Sep 09 16:26:43.556955 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:43.556935    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd73e10d-18bb-42fd-be79-ae6949f19f36-tls-certs\") pod \"test-model-6d44b6-kserve-67d6d975d9-xw82r\" (UID: \"dd73e10d-18bb-42fd-be79-ae6949f19f36\") " pod="ai-tenant-e2e-inf-a-6d44b6/test-model-6d44b6-kserve-67d6d975d9-xw82r"
Sep 09 16:26:43.562486 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:43.562432    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgmfb\" (UniqueName: \"kubernetes.io/projected/dd73e10d-18bb-42fd-be79-ae6949f19f36-kube-api-access-xgmfb\") pod \"test-model-6d44b6-kserve-67d6d975d9-xw82r\" (UID: \"dd73e10d-18bb-42fd-be79-ae6949f19f36\") " pod="ai-tenant-e2e-inf-a-6d44b6/test-model-6d44b6-kserve-67d6d975d9-xw82r"
Sep 09 16:26:43.620805 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:43.620760    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-6d44b6/test-model-6d44b6-kserve-67d6d975d9-xw82r"
Sep 09 16:26:43.901784 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:43.901739    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-6d44b6/test-model-6d44b6-kserve-67d6d975d9-xw82r"]
Sep 09 16:26:43.904896 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:26:43.904868    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddd73e10d_18bb_42fd_be79_ae6949f19f36.slice/crio-749190ebe038a434b9a1f39aa55e3c89a04380b349d6fbbbeb5bcb34acd9abe7 WatchSource:0}: Error finding container 749190ebe038a434b9a1f39aa55e3c89a04380b349d6fbbbeb5bcb34acd9abe7: Status 404 returned error can't find the container with id 749190ebe038a434b9a1f39aa55e3c89a04380b349d6fbbbeb5bcb34acd9abe7
Sep 09 16:26:44.536860 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:44.536815    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-6d44b6/test-model-6d44b6-kserve-67d6d975d9-xw82r" event={"ID":"dd73e10d-18bb-42fd-be79-ae6949f19f36","Type":"ContainerStarted","Data":"290736c36ea54c4a4ec869a74e67158bc387e4d6f9480168f098853c662705d6"}
Sep 09 16:26:44.537329 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:44.536874    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-6d44b6/test-model-6d44b6-kserve-67d6d975d9-xw82r" event={"ID":"dd73e10d-18bb-42fd-be79-ae6949f19f36","Type":"ContainerStarted","Data":"749190ebe038a434b9a1f39aa55e3c89a04380b349d6fbbbeb5bcb34acd9abe7"}
Sep 09 16:26:44.552332 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:44.552155    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-6d44b6/test-model-6d44b6-kserve-67d6d975d9-xw82r" podStartSLOduration=1.552135631 podStartE2EDuration="1.552135631s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 16:26:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 16:26:44.551760782 +0000 UTC m=+1079.969610782" watchObservedRunningTime="2026-09-09 16:26:44.552135631 +0000 UTC m=+1079.969985632"
Sep 09 16:26:45.377332 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:45.376891    2567 patch_prober.go:28] interesting pod/e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": read tcp 10.134.0.2:33736->10.134.0.36:15021: read: connection reset by peer" start-of-body=
Sep 09 16:26:45.377332 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:45.376961    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh" podUID="1902f142-3560-4d43-94d0-f8041d1eb0b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": read tcp 10.134.0.2:33736->10.134.0.36:15021: read: connection reset by peer"
Sep 09 16:26:45.543218 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:45.543129    2567 generic.go:356] "Generic (PLEG): container finished" podID="1902f142-3560-4d43-94d0-f8041d1eb0b3" containerID="a682f7bf7af555d9f16dbee55b829aef329a4392e085eeddc384357054fa427d" exitCode=0
Sep 09 16:26:45.543218 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:45.543204    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh" event={"ID":"1902f142-3560-4d43-94d0-f8041d1eb0b3","Type":"ContainerDied","Data":"a682f7bf7af555d9f16dbee55b829aef329a4392e085eeddc384357054fa427d"}
Sep 09 16:26:45.543734 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:45.543718    2567 scope.go:117] "RemoveContainer" containerID="a682f7bf7af555d9f16dbee55b829aef329a4392e085eeddc384357054fa427d"
Sep 09 16:26:45.544801 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:45.544767    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:26:45.544860 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:45.544845    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:26:45.544896 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:45.544881    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:26:46.551430 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:46.551392    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh" event={"ID":"1902f142-3560-4d43-94d0-f8041d1eb0b3","Type":"ContainerStarted","Data":"c58660dd9749dc22a09158a8a725b83a7f31a2b9ea33e9c1401534797c7a3762"}
Sep 09 16:26:46.906384 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:46.906347    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh"
Sep 09 16:26:46.906830 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:46.906738    2567 patch_prober.go:28] interesting pod/e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Sep 09 16:26:46.907024 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:46.906999    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh" podUID="1902f142-3560-4d43-94d0-f8041d1eb0b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Sep 09 16:26:47.906644 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:47.906599    2567 patch_prober.go:28] interesting pod/e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Sep 09 16:26:47.907158 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:47.906683    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh" podUID="1902f142-3560-4d43-94d0-f8041d1eb0b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Sep 09 16:26:48.906947 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:48.906903    2567 patch_prober.go:28] interesting pod/e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Sep 09 16:26:48.907574 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:48.907536    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh" podUID="1902f142-3560-4d43-94d0-f8041d1eb0b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Sep 09 16:26:49.906736 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:49.906692    2567 patch_prober.go:28] interesting pod/e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Sep 09 16:26:49.907065 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:49.907031    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh" podUID="1902f142-3560-4d43-94d0-f8041d1eb0b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Sep 09 16:26:50.420878 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:50.420826    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-743aed-qss2d"]
Sep 09 16:26:50.421482 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:50.421434    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5749d6e6-548c-40af-928a-7f8471c7492f" containerName="istio-proxy"
Sep 09 16:26:50.421482 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:50.421477    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="5749d6e6-548c-40af-928a-7f8471c7492f" containerName="istio-proxy"
Sep 09 16:26:50.421680 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:50.421627    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="5749d6e6-548c-40af-928a-7f8471c7492f" containerName="istio-proxy"
Sep 09 16:26:50.425358 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:50.425333    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-743aed-qss2d"
Sep 09 16:26:50.427629 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:50.427607    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 09 16:26:50.427802 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:50.427741    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wxghs\""
Sep 09 16:26:50.429768 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:50.429706    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-743aed-qss2d"]
Sep 09 16:26:50.534596 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:50.534558    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea6a4010-a093-47fe-acdd-21f3dc4bf8da-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-743aed-qss2d\" (UID: \"ea6a4010-a093-47fe-acdd-21f3dc4bf8da\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-743aed-qss2d"
Sep 09 16:26:50.636083 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:50.636043    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea6a4010-a093-47fe-acdd-21f3dc4bf8da-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-743aed-qss2d\" (UID: \"ea6a4010-a093-47fe-acdd-21f3dc4bf8da\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-743aed-qss2d"
Sep 09 16:26:50.636715 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:50.636694    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea6a4010-a093-47fe-acdd-21f3dc4bf8da-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-743aed-qss2d\" (UID: \"ea6a4010-a093-47fe-acdd-21f3dc4bf8da\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-743aed-qss2d"
Sep 09 16:26:50.739939 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:50.739846    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-743aed-qss2d"
Sep 09 16:26:50.911398 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:50.910915    2567 patch_prober.go:28] interesting pod/e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Sep 09 16:26:50.911398 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:50.910980    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh" podUID="1902f142-3560-4d43-94d0-f8041d1eb0b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Sep 09 16:26:51.022342 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:51.022065    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-743aed-qss2d"]
Sep 09 16:26:51.024614 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:26:51.024579    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea6a4010_a093_47fe_acdd_21f3dc4bf8da.slice/crio-51981665921a3b598a944d25784cba75ba11e414d2d9f1709201316c93d1605c WatchSource:0}: Error finding container 51981665921a3b598a944d25784cba75ba11e414d2d9f1709201316c93d1605c: Status 404 returned error can't find the container with id 51981665921a3b598a944d25784cba75ba11e414d2d9f1709201316c93d1605c
Sep 09 16:26:51.578355 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:51.578321    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-743aed-qss2d" event={"ID":"ea6a4010-a093-47fe-acdd-21f3dc4bf8da","Type":"ContainerStarted","Data":"ec6a444c756149d0e61b70a04538b0bc284b9d9d39da5c6112b8ec3e2bfcf625"}
Sep 09 16:26:51.578603 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:51.578386    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-743aed-qss2d" event={"ID":"ea6a4010-a093-47fe-acdd-21f3dc4bf8da","Type":"ContainerStarted","Data":"51981665921a3b598a944d25784cba75ba11e414d2d9f1709201316c93d1605c"}
Sep 09 16:26:52.130814 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:52.130783    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh"
Sep 09 16:26:52.131305 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:52.131190    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh"
Sep 09 16:26:52.132754 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:52.132731    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh"
Sep 09 16:26:52.150818 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:52.150715    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-743aed-qss2d" podStartSLOduration=2.150701155 podStartE2EDuration="2.150701155s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 16:26:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 16:26:51.595731253 +0000 UTC m=+1087.013581253" watchObservedRunningTime="2026-09-09 16:26:52.150701155 +0000 UTC m=+1087.568551154"
Sep 09 16:26:52.584821 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:52.584783    2567 generic.go:356] "Generic (PLEG): container finished" podID="ea6a4010-a093-47fe-acdd-21f3dc4bf8da" containerID="ec6a444c756149d0e61b70a04538b0bc284b9d9d39da5c6112b8ec3e2bfcf625" exitCode=0
Sep 09 16:26:52.584998 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:52.584870    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-743aed-qss2d" event={"ID":"ea6a4010-a093-47fe-acdd-21f3dc4bf8da","Type":"ContainerDied","Data":"ec6a444c756149d0e61b70a04538b0bc284b9d9d39da5c6112b8ec3e2bfcf625"}
Sep 09 16:26:53.621694 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:53.621660    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-6d44b6/test-model-6d44b6-kserve-67d6d975d9-xw82r"
Sep 09 16:26:53.622195 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:53.621716    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-6d44b6/test-model-6d44b6-kserve-67d6d975d9-xw82r"
Sep 09 16:26:53.634519 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:53.634485    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-6d44b6/test-model-6d44b6-kserve-67d6d975d9-xw82r"
Sep 09 16:26:53.715584 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:53.715560    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:26:53.730069 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:53.730050    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-743aed-qss2d"
Sep 09 16:26:53.870507 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:53.870379    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea6a4010-a093-47fe-acdd-21f3dc4bf8da-maas-api-service-ca\") pod \"ea6a4010-a093-47fe-acdd-21f3dc4bf8da\" (UID: \"ea6a4010-a093-47fe-acdd-21f3dc4bf8da\") "
Sep 09 16:26:53.870842 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:53.870808    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea6a4010-a093-47fe-acdd-21f3dc4bf8da-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ea6a4010-a093-47fe-acdd-21f3dc4bf8da" (UID: "ea6a4010-a093-47fe-acdd-21f3dc4bf8da"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 16:26:53.971636 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:53.971598    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea6a4010-a093-47fe-acdd-21f3dc4bf8da-maas-api-service-ca\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:26:54.596512 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:54.596451    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-743aed-qss2d"
Sep 09 16:26:54.596665 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:54.596481    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-743aed-qss2d" event={"ID":"ea6a4010-a093-47fe-acdd-21f3dc4bf8da","Type":"ContainerDied","Data":"51981665921a3b598a944d25784cba75ba11e414d2d9f1709201316c93d1605c"}
Sep 09 16:26:54.596665 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:54.596598    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="51981665921a3b598a944d25784cba75ba11e414d2d9f1709201316c93d1605c"
Sep 09 16:26:54.608365 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:54.608333    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-6d44b6/test-model-6d44b6-kserve-67d6d975d9-xw82r"
Sep 09 16:26:54.737486 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:54.737434    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8a375fa0-nlhvj"]
Sep 09 16:26:54.738116 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:54.738089    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea6a4010-a093-47fe-acdd-21f3dc4bf8da" containerName="revoke-keys"
Sep 09 16:26:54.738116 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:54.738119    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea6a4010-a093-47fe-acdd-21f3dc4bf8da" containerName="revoke-keys"
Sep 09 16:26:54.738289 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:54.738214    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea6a4010-a093-47fe-acdd-21f3dc4bf8da" containerName="revoke-keys"
Sep 09 16:26:54.741987 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:54.741963    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8a375fa0-nlhvj"
Sep 09 16:26:54.744943 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:54.744920    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wxghs\""
Sep 09 16:26:54.748470 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:54.748434    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 09 16:26:54.757072 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:54.757047    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8a375fa0-nlhvj"]
Sep 09 16:26:54.880841 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:54.880756    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4b7aac7-7f68-4a90-81b4-618708db2f46-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-8a375fa0-nlhvj\" (UID: \"d4b7aac7-7f68-4a90-81b4-618708db2f46\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8a375fa0-nlhvj"
Sep 09 16:26:54.981982 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:54.981949    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4b7aac7-7f68-4a90-81b4-618708db2f46-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-8a375fa0-nlhvj\" (UID: \"d4b7aac7-7f68-4a90-81b4-618708db2f46\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8a375fa0-nlhvj"
Sep 09 16:26:54.982620 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:54.982603    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4b7aac7-7f68-4a90-81b4-618708db2f46-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-8a375fa0-nlhvj\" (UID: \"d4b7aac7-7f68-4a90-81b4-618708db2f46\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8a375fa0-nlhvj"
Sep 09 16:26:55.052522 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:55.052493    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8a375fa0-nlhvj"
Sep 09 16:26:55.320140 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:55.320113    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8a375fa0-nlhvj"]
Sep 09 16:26:55.322199 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:26:55.322173    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4b7aac7_7f68_4a90_81b4_618708db2f46.slice/crio-06ae113df1634342f15f5ed0c8a2ea9c36b5d9cacb8b99d25c4af7caf422ff8f WatchSource:0}: Error finding container 06ae113df1634342f15f5ed0c8a2ea9c36b5d9cacb8b99d25c4af7caf422ff8f: Status 404 returned error can't find the container with id 06ae113df1634342f15f5ed0c8a2ea9c36b5d9cacb8b99d25c4af7caf422ff8f
Sep 09 16:26:55.603947 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:55.603914    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8a375fa0-nlhvj" event={"ID":"d4b7aac7-7f68-4a90-81b4-618708db2f46","Type":"ContainerStarted","Data":"06ae113df1634342f15f5ed0c8a2ea9c36b5d9cacb8b99d25c4af7caf422ff8f"}
Sep 09 16:26:56.611229 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:56.611190    2567 generic.go:356] "Generic (PLEG): container finished" podID="d4b7aac7-7f68-4a90-81b4-618708db2f46" containerID="876c837b7708cc91ec440fbfb9a2eac37df9081fd1202d83164589bae3a92fef" exitCode=0
Sep 09 16:26:56.611669 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:56.611246    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8a375fa0-nlhvj" event={"ID":"d4b7aac7-7f68-4a90-81b4-618708db2f46","Type":"ContainerDied","Data":"876c837b7708cc91ec440fbfb9a2eac37df9081fd1202d83164589bae3a92fef"}
Sep 09 16:26:57.758094 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:57.758070    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8a375fa0-nlhvj"
Sep 09 16:26:57.809127 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:57.809088    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4b7aac7-7f68-4a90-81b4-618708db2f46-maas-api-service-ca\") pod \"d4b7aac7-7f68-4a90-81b4-618708db2f46\" (UID: \"d4b7aac7-7f68-4a90-81b4-618708db2f46\") "
Sep 09 16:26:57.809449 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:57.809426    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4b7aac7-7f68-4a90-81b4-618708db2f46-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d4b7aac7-7f68-4a90-81b4-618708db2f46" (UID: "d4b7aac7-7f68-4a90-81b4-618708db2f46"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 16:26:57.910839 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:57.910761    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4b7aac7-7f68-4a90-81b4-618708db2f46-maas-api-service-ca\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:26:58.622914 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:58.622882    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8a375fa0-nlhvj" event={"ID":"d4b7aac7-7f68-4a90-81b4-618708db2f46","Type":"ContainerDied","Data":"06ae113df1634342f15f5ed0c8a2ea9c36b5d9cacb8b99d25c4af7caf422ff8f"}
Sep 09 16:26:58.622914 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:58.622909    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8a375fa0-nlhvj"
Sep 09 16:26:58.623124 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:26:58.622927    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="06ae113df1634342f15f5ed0c8a2ea9c36b5d9cacb8b99d25c4af7caf422ff8f"
Sep 09 16:27:00.780448 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:00.780408    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-743aed-qss2d"]
Sep 09 16:27:00.782325 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:00.782299    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-743aed-qss2d"]
Sep 09 16:27:00.918723 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:00.918678    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8a375fa0-nlhvj"]
Sep 09 16:27:00.920320 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:00.920296    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8a375fa0-nlhvj"]
Sep 09 16:27:01.041028 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:01.040941    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4b7aac7-7f68-4a90-81b4-618708db2f46" path="/var/lib/kubelet/pods/d4b7aac7-7f68-4a90-81b4-618708db2f46/volumes"
Sep 09 16:27:01.042394 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:01.042366    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea6a4010-a093-47fe-acdd-21f3dc4bf8da" path="/var/lib/kubelet/pods/ea6a4010-a093-47fe-acdd-21f3dc4bf8da/volumes"
Sep 09 16:27:02.168472 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:02.168417    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh"]
Sep 09 16:27:02.168872 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:02.168702    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh" podUID="1902f142-3560-4d43-94d0-f8041d1eb0b3" containerName="istio-proxy" containerID="cri-o://c58660dd9749dc22a09158a8a725b83a7f31a2b9ea33e9c1401534797c7a3762" gracePeriod=30
Sep 09 16:27:07.134008 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:07.133972    2567 patch_prober.go:28] interesting pod/e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 09 16:27:07.134588 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:07.134052    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh" podUID="1902f142-3560-4d43-94d0-f8041d1eb0b3" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 16:27:07.446760 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:07.446735    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh"
Sep 09 16:27:07.515987 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:07.515956    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1902f142-3560-4d43-94d0-f8041d1eb0b3-credential-socket\") pod \"1902f142-3560-4d43-94d0-f8041d1eb0b3\" (UID: \"1902f142-3560-4d43-94d0-f8041d1eb0b3\") "
Sep 09 16:27:07.516185 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:07.516029    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1902f142-3560-4d43-94d0-f8041d1eb0b3-workload-socket\") pod \"1902f142-3560-4d43-94d0-f8041d1eb0b3\" (UID: \"1902f142-3560-4d43-94d0-f8041d1eb0b3\") "
Sep 09 16:27:07.516185 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:07.516069    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1902f142-3560-4d43-94d0-f8041d1eb0b3-istio-data\") pod \"1902f142-3560-4d43-94d0-f8041d1eb0b3\" (UID: \"1902f142-3560-4d43-94d0-f8041d1eb0b3\") "
Sep 09 16:27:07.516185 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:07.516101    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1902f142-3560-4d43-94d0-f8041d1eb0b3-istio-token\") pod \"1902f142-3560-4d43-94d0-f8041d1eb0b3\" (UID: \"1902f142-3560-4d43-94d0-f8041d1eb0b3\") "
Sep 09 16:27:07.516355 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:07.516194    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1902f142-3560-4d43-94d0-f8041d1eb0b3-istiod-ca-cert\") pod \"1902f142-3560-4d43-94d0-f8041d1eb0b3\" (UID: \"1902f142-3560-4d43-94d0-f8041d1eb0b3\") "
Sep 09 16:27:07.516355 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:07.516211    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1902f142-3560-4d43-94d0-f8041d1eb0b3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1902f142-3560-4d43-94d0-f8041d1eb0b3" (UID: "1902f142-3560-4d43-94d0-f8041d1eb0b3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:27:07.516355 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:07.516247    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1902f142-3560-4d43-94d0-f8041d1eb0b3-istio-podinfo\") pod \"1902f142-3560-4d43-94d0-f8041d1eb0b3\" (UID: \"1902f142-3560-4d43-94d0-f8041d1eb0b3\") "
Sep 09 16:27:07.516355 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:07.516277    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jmht7\" (UniqueName: \"kubernetes.io/projected/1902f142-3560-4d43-94d0-f8041d1eb0b3-kube-api-access-jmht7\") pod \"1902f142-3560-4d43-94d0-f8041d1eb0b3\" (UID: \"1902f142-3560-4d43-94d0-f8041d1eb0b3\") "
Sep 09 16:27:07.516355 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:07.516311    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1902f142-3560-4d43-94d0-f8041d1eb0b3-workload-certs\") pod \"1902f142-3560-4d43-94d0-f8041d1eb0b3\" (UID: \"1902f142-3560-4d43-94d0-f8041d1eb0b3\") "
Sep 09 16:27:07.516355 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:07.516332    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1902f142-3560-4d43-94d0-f8041d1eb0b3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1902f142-3560-4d43-94d0-f8041d1eb0b3" (UID: "1902f142-3560-4d43-94d0-f8041d1eb0b3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:27:07.516355 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:07.516345    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1902f142-3560-4d43-94d0-f8041d1eb0b3-istio-envoy\") pod \"1902f142-3560-4d43-94d0-f8041d1eb0b3\" (UID: \"1902f142-3560-4d43-94d0-f8041d1eb0b3\") "
Sep 09 16:27:07.516747 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:07.516351    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1902f142-3560-4d43-94d0-f8041d1eb0b3-istio-data" (OuterVolumeSpecName: "istio-data") pod "1902f142-3560-4d43-94d0-f8041d1eb0b3" (UID: "1902f142-3560-4d43-94d0-f8041d1eb0b3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:27:07.516747 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:07.516616    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1902f142-3560-4d43-94d0-f8041d1eb0b3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1902f142-3560-4d43-94d0-f8041d1eb0b3" (UID: "1902f142-3560-4d43-94d0-f8041d1eb0b3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 16:27:07.516747 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:07.516705    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1902f142-3560-4d43-94d0-f8041d1eb0b3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1902f142-3560-4d43-94d0-f8041d1eb0b3" (UID: "1902f142-3560-4d43-94d0-f8041d1eb0b3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:27:07.516924 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:07.516863    2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1902f142-3560-4d43-94d0-f8041d1eb0b3-workload-socket\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:27:07.516924 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:07.516877    2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1902f142-3560-4d43-94d0-f8041d1eb0b3-istio-data\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:27:07.516924 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:07.516886    2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1902f142-3560-4d43-94d0-f8041d1eb0b3-istiod-ca-cert\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:27:07.516924 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:07.516894    2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1902f142-3560-4d43-94d0-f8041d1eb0b3-workload-certs\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:27:07.516924 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:07.516901    2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1902f142-3560-4d43-94d0-f8041d1eb0b3-credential-socket\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:27:07.518723 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:07.518663    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1902f142-3560-4d43-94d0-f8041d1eb0b3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1902f142-3560-4d43-94d0-f8041d1eb0b3" (UID: "1902f142-3560-4d43-94d0-f8041d1eb0b3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:27:07.518723 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:07.518678    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1902f142-3560-4d43-94d0-f8041d1eb0b3-istio-token" (OuterVolumeSpecName: "istio-token") pod "1902f142-3560-4d43-94d0-f8041d1eb0b3" (UID: "1902f142-3560-4d43-94d0-f8041d1eb0b3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 16:27:07.519086 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:07.519063    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1902f142-3560-4d43-94d0-f8041d1eb0b3-kube-api-access-jmht7" (OuterVolumeSpecName: "kube-api-access-jmht7") pod "1902f142-3560-4d43-94d0-f8041d1eb0b3" (UID: "1902f142-3560-4d43-94d0-f8041d1eb0b3"). InnerVolumeSpecName "kube-api-access-jmht7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 16:27:07.519137 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:07.519075    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1902f142-3560-4d43-94d0-f8041d1eb0b3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1902f142-3560-4d43-94d0-f8041d1eb0b3" (UID: "1902f142-3560-4d43-94d0-f8041d1eb0b3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 16:27:07.617716 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:07.617676    2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1902f142-3560-4d43-94d0-f8041d1eb0b3-istio-token\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:27:07.617716 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:07.617706    2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1902f142-3560-4d43-94d0-f8041d1eb0b3-istio-podinfo\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:27:07.617716 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:07.617718    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jmht7\" (UniqueName: \"kubernetes.io/projected/1902f142-3560-4d43-94d0-f8041d1eb0b3-kube-api-access-jmht7\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:27:07.617963 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:07.617732    2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1902f142-3560-4d43-94d0-f8041d1eb0b3-istio-envoy\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:27:07.672990 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:07.672954    2567 generic.go:356] "Generic (PLEG): container finished" podID="1902f142-3560-4d43-94d0-f8041d1eb0b3" containerID="c58660dd9749dc22a09158a8a725b83a7f31a2b9ea33e9c1401534797c7a3762" exitCode=0
Sep 09 16:27:07.673144 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:07.673003    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh"
Sep 09 16:27:07.673144 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:07.673026    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh" event={"ID":"1902f142-3560-4d43-94d0-f8041d1eb0b3","Type":"ContainerDied","Data":"c58660dd9749dc22a09158a8a725b83a7f31a2b9ea33e9c1401534797c7a3762"}
Sep 09 16:27:07.673144 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:07.673071    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh" event={"ID":"1902f142-3560-4d43-94d0-f8041d1eb0b3","Type":"ContainerDied","Data":"9c191de3d3a615504f99694ea5c28f3e784c1f2a9f98ee6f0b56b29214e6a058"}
Sep 09 16:27:07.673144 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:07.673094    2567 scope.go:117] "RemoveContainer" containerID="c58660dd9749dc22a09158a8a725b83a7f31a2b9ea33e9c1401534797c7a3762"
Sep 09 16:27:07.682469 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:07.682431    2567 scope.go:117] "RemoveContainer" containerID="a682f7bf7af555d9f16dbee55b829aef329a4392e085eeddc384357054fa427d"
Sep 09 16:27:07.690527 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:07.690499    2567 scope.go:117] "RemoveContainer" containerID="c58660dd9749dc22a09158a8a725b83a7f31a2b9ea33e9c1401534797c7a3762"
Sep 09 16:27:07.690854 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:27:07.690788    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c58660dd9749dc22a09158a8a725b83a7f31a2b9ea33e9c1401534797c7a3762\": container with ID starting with c58660dd9749dc22a09158a8a725b83a7f31a2b9ea33e9c1401534797c7a3762 not found: ID does not exist" containerID="c58660dd9749dc22a09158a8a725b83a7f31a2b9ea33e9c1401534797c7a3762"
Sep 09 16:27:07.690854 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:07.690828    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c58660dd9749dc22a09158a8a725b83a7f31a2b9ea33e9c1401534797c7a3762"} err="failed to get container status \"c58660dd9749dc22a09158a8a725b83a7f31a2b9ea33e9c1401534797c7a3762\": rpc error: code = NotFound desc = could not find container \"c58660dd9749dc22a09158a8a725b83a7f31a2b9ea33e9c1401534797c7a3762\": container with ID starting with c58660dd9749dc22a09158a8a725b83a7f31a2b9ea33e9c1401534797c7a3762 not found: ID does not exist"
Sep 09 16:27:07.691066 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:07.690977    2567 scope.go:117] "RemoveContainer" containerID="a682f7bf7af555d9f16dbee55b829aef329a4392e085eeddc384357054fa427d"
Sep 09 16:27:07.691419 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:27:07.691358    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a682f7bf7af555d9f16dbee55b829aef329a4392e085eeddc384357054fa427d\": container with ID starting with a682f7bf7af555d9f16dbee55b829aef329a4392e085eeddc384357054fa427d not found: ID does not exist" containerID="a682f7bf7af555d9f16dbee55b829aef329a4392e085eeddc384357054fa427d"
Sep 09 16:27:07.691419 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:07.691386    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a682f7bf7af555d9f16dbee55b829aef329a4392e085eeddc384357054fa427d"} err="failed to get container status \"a682f7bf7af555d9f16dbee55b829aef329a4392e085eeddc384357054fa427d\": rpc error: code = NotFound desc = could not find container \"a682f7bf7af555d9f16dbee55b829aef329a4392e085eeddc384357054fa427d\": container with ID starting with a682f7bf7af555d9f16dbee55b829aef329a4392e085eeddc384357054fa427d not found: ID does not exist"
Sep 09 16:27:07.693819 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:07.693798    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh"]
Sep 09 16:27:07.698069 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:07.698049    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-8a375fa0-openshift-default-7b58696b98-phjjh"]
Sep 09 16:27:08.470528 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:08.470494    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-743aed-openshift-default-658d874dc4-lzxsj"]
Sep 09 16:27:08.470955 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:08.470794    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-autores-743aed-openshift-default-658d874dc4-lzxsj" podUID="2b6f1200-474b-4101-b718-2b2d0cdd490a" containerName="istio-proxy" containerID="cri-o://7f1d5470d42639ebd84df7dd2a9044de67d9eaac7124a4f43e899b86c237895f" gracePeriod=30
Sep 09 16:27:08.551111 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:08.551078    2567 patch_prober.go:28] interesting pod/e2e-autores-743aed-openshift-default-658d874dc4-lzxsj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 09 16:27:08.551245 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:08.551139    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-autores-743aed-openshift-default-658d874dc4-lzxsj" podUID="2b6f1200-474b-4101-b718-2b2d0cdd490a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 16:27:09.041043 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:09.041011    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1902f142-3560-4d43-94d0-f8041d1eb0b3" path="/var/lib/kubelet/pods/1902f142-3560-4d43-94d0-f8041d1eb0b3/volumes"
Sep 09 16:27:09.249447 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:09.249415    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5"]
Sep 09 16:27:09.249910 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:09.249892    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1902f142-3560-4d43-94d0-f8041d1eb0b3" containerName="istio-proxy"
Sep 09 16:27:09.249998 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:09.249910    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="1902f142-3560-4d43-94d0-f8041d1eb0b3" containerName="istio-proxy"
Sep 09 16:27:09.249998 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:09.249926    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4b7aac7-7f68-4a90-81b4-618708db2f46" containerName="revoke-keys"
Sep 09 16:27:09.249998 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:09.249933    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4b7aac7-7f68-4a90-81b4-618708db2f46" containerName="revoke-keys"
Sep 09 16:27:09.250117 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:09.250004    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="1902f142-3560-4d43-94d0-f8041d1eb0b3" containerName="istio-proxy"
Sep 09 16:27:09.250117 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:09.250016    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4b7aac7-7f68-4a90-81b4-618708db2f46" containerName="revoke-keys"
Sep 09 16:27:09.254733 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:09.254713    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5"
Sep 09 16:27:09.256852 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:09.256832    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-explicit-3ad4f1-openshift-default-dockercfg-jltnw\""
Sep 09 16:27:09.264032 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:09.264005    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5"]
Sep 09 16:27:09.338069 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:09.338030    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc3a0139-4de3-4798-ab06-1ee71ff29ca3-workload-socket\") pod \"e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5\" (UID: \"fc3a0139-4de3-4798-ab06-1ee71ff29ca3\") " pod="openshift-ingress/e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5"
Sep 09 16:27:09.338277 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:09.338079    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc3a0139-4de3-4798-ab06-1ee71ff29ca3-credential-socket\") pod \"e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5\" (UID: \"fc3a0139-4de3-4798-ab06-1ee71ff29ca3\") " pod="openshift-ingress/e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5"
Sep 09 16:27:09.338277 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:09.338103    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc3a0139-4de3-4798-ab06-1ee71ff29ca3-istio-data\") pod \"e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5\" (UID: \"fc3a0139-4de3-4798-ab06-1ee71ff29ca3\") " pod="openshift-ingress/e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5"
Sep 09 16:27:09.338277 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:09.338137    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc3a0139-4de3-4798-ab06-1ee71ff29ca3-istiod-ca-cert\") pod \"e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5\" (UID: \"fc3a0139-4de3-4798-ab06-1ee71ff29ca3\") " pod="openshift-ingress/e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5"
Sep 09 16:27:09.338277 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:09.338173    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc3a0139-4de3-4798-ab06-1ee71ff29ca3-istio-token\") pod \"e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5\" (UID: \"fc3a0139-4de3-4798-ab06-1ee71ff29ca3\") " pod="openshift-ingress/e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5"
Sep 09 16:27:09.338277 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:09.338268    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc3a0139-4de3-4798-ab06-1ee71ff29ca3-istio-envoy\") pod \"e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5\" (UID: \"fc3a0139-4de3-4798-ab06-1ee71ff29ca3\") " pod="openshift-ingress/e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5"
Sep 09 16:27:09.338606 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:09.338357    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6pdlx\" (UniqueName: \"kubernetes.io/projected/fc3a0139-4de3-4798-ab06-1ee71ff29ca3-kube-api-access-6pdlx\") pod \"e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5\" (UID: \"fc3a0139-4de3-4798-ab06-1ee71ff29ca3\") " pod="openshift-ingress/e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5"
Sep 09 16:27:09.338606 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:09.338384    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc3a0139-4de3-4798-ab06-1ee71ff29ca3-workload-certs\") pod \"e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5\" (UID: \"fc3a0139-4de3-4798-ab06-1ee71ff29ca3\") " pod="openshift-ingress/e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5"
Sep 09 16:27:09.338606 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:09.338425    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc3a0139-4de3-4798-ab06-1ee71ff29ca3-istio-podinfo\") pod \"e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5\" (UID: \"fc3a0139-4de3-4798-ab06-1ee71ff29ca3\") " pod="openshift-ingress/e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5"
Sep 09 16:27:09.439682 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:09.439641    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc3a0139-4de3-4798-ab06-1ee71ff29ca3-istiod-ca-cert\") pod \"e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5\" (UID: \"fc3a0139-4de3-4798-ab06-1ee71ff29ca3\") " pod="openshift-ingress/e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5"
Sep 09 16:27:09.439876 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:09.439694    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc3a0139-4de3-4798-ab06-1ee71ff29ca3-istio-token\") pod \"e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5\" (UID: \"fc3a0139-4de3-4798-ab06-1ee71ff29ca3\") " pod="openshift-ingress/e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5"
Sep 09 16:27:09.439876 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:09.439738    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc3a0139-4de3-4798-ab06-1ee71ff29ca3-istio-envoy\") pod \"e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5\" (UID: \"fc3a0139-4de3-4798-ab06-1ee71ff29ca3\") " pod="openshift-ingress/e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5"
Sep 09 16:27:09.439876 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:09.439831    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pdlx\" (UniqueName: \"kubernetes.io/projected/fc3a0139-4de3-4798-ab06-1ee71ff29ca3-kube-api-access-6pdlx\") pod \"e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5\" (UID: \"fc3a0139-4de3-4798-ab06-1ee71ff29ca3\") " pod="openshift-ingress/e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5"
Sep 09 16:27:09.439876 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:09.439857    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc3a0139-4de3-4798-ab06-1ee71ff29ca3-workload-certs\") pod \"e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5\" (UID: \"fc3a0139-4de3-4798-ab06-1ee71ff29ca3\") " pod="openshift-ingress/e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5"
Sep 09 16:27:09.440104 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:09.439890    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc3a0139-4de3-4798-ab06-1ee71ff29ca3-istio-podinfo\") pod \"e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5\" (UID: \"fc3a0139-4de3-4798-ab06-1ee71ff29ca3\") " pod="openshift-ingress/e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5"
Sep 09 16:27:09.440104 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:09.439937    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc3a0139-4de3-4798-ab06-1ee71ff29ca3-workload-socket\") pod \"e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5\" (UID: \"fc3a0139-4de3-4798-ab06-1ee71ff29ca3\") " pod="openshift-ingress/e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5"
Sep 09 16:27:09.440104 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:09.439975    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc3a0139-4de3-4798-ab06-1ee71ff29ca3-credential-socket\") pod \"e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5\" (UID: \"fc3a0139-4de3-4798-ab06-1ee71ff29ca3\") " pod="openshift-ingress/e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5"
Sep 09 16:27:09.440104 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:09.440001    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc3a0139-4de3-4798-ab06-1ee71ff29ca3-istio-data\") pod \"e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5\" (UID: \"fc3a0139-4de3-4798-ab06-1ee71ff29ca3\") " pod="openshift-ingress/e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5"
Sep 09 16:27:09.440302 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:09.440281    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc3a0139-4de3-4798-ab06-1ee71ff29ca3-workload-certs\") pod \"e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5\" (UID: \"fc3a0139-4de3-4798-ab06-1ee71ff29ca3\") " pod="openshift-ingress/e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5"
Sep 09 16:27:09.440355 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:09.440308    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc3a0139-4de3-4798-ab06-1ee71ff29ca3-workload-socket\") pod \"e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5\" (UID: \"fc3a0139-4de3-4798-ab06-1ee71ff29ca3\") " pod="openshift-ingress/e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5"
Sep 09 16:27:09.440394 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:09.440363    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc3a0139-4de3-4798-ab06-1ee71ff29ca3-istio-data\") pod \"e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5\" (UID: \"fc3a0139-4de3-4798-ab06-1ee71ff29ca3\") " pod="openshift-ingress/e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5"
Sep 09 16:27:09.440515 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:09.440499    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc3a0139-4de3-4798-ab06-1ee71ff29ca3-istiod-ca-cert\") pod \"e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5\" (UID: \"fc3a0139-4de3-4798-ab06-1ee71ff29ca3\") " pod="openshift-ingress/e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5"
Sep 09 16:27:09.440587 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:09.440562    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc3a0139-4de3-4798-ab06-1ee71ff29ca3-credential-socket\") pod \"e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5\" (UID: \"fc3a0139-4de3-4798-ab06-1ee71ff29ca3\") " pod="openshift-ingress/e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5"
Sep 09 16:27:09.442237 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:09.442216    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc3a0139-4de3-4798-ab06-1ee71ff29ca3-istio-envoy\") pod \"e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5\" (UID: \"fc3a0139-4de3-4798-ab06-1ee71ff29ca3\") " pod="openshift-ingress/e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5"
Sep 09 16:27:09.442507 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:09.442484    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc3a0139-4de3-4798-ab06-1ee71ff29ca3-istio-podinfo\") pod \"e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5\" (UID: \"fc3a0139-4de3-4798-ab06-1ee71ff29ca3\") " pod="openshift-ingress/e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5"
Sep 09 16:27:09.455738 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:09.455708    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc3a0139-4de3-4798-ab06-1ee71ff29ca3-istio-token\") pod \"e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5\" (UID: \"fc3a0139-4de3-4798-ab06-1ee71ff29ca3\") " pod="openshift-ingress/e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5"
Sep 09 16:27:09.455862 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:09.455844    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6pdlx\" (UniqueName: \"kubernetes.io/projected/fc3a0139-4de3-4798-ab06-1ee71ff29ca3-kube-api-access-6pdlx\") pod \"e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5\" (UID: \"fc3a0139-4de3-4798-ab06-1ee71ff29ca3\") " pod="openshift-ingress/e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5"
Sep 09 16:27:09.566263 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:09.566222    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5"
Sep 09 16:27:09.850251 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:09.850225    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5"]
Sep 09 16:27:09.852161 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:27:09.852131    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc3a0139_4de3_4798_ab06_1ee71ff29ca3.slice/crio-fd85235ddd37ce93818c34f19857d08ea357b7c13a2c6ec5de87691ba54b949e WatchSource:0}: Error finding container fd85235ddd37ce93818c34f19857d08ea357b7c13a2c6ec5de87691ba54b949e: Status 404 returned error can't find the container with id fd85235ddd37ce93818c34f19857d08ea357b7c13a2c6ec5de87691ba54b949e
Sep 09 16:27:09.854326 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:09.854295    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:27:09.854381 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:09.854359    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:27:09.854414 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:09.854391    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:27:10.690856 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:10.690819    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5" event={"ID":"fc3a0139-4de3-4798-ab06-1ee71ff29ca3","Type":"ContainerStarted","Data":"f5b7d8608c786adb0f387e0949d7f66e07534a9c5785777a8e67d97f29c7caa1"}
Sep 09 16:27:10.691303 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:10.690874    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5" event={"ID":"fc3a0139-4de3-4798-ab06-1ee71ff29ca3","Type":"ContainerStarted","Data":"fd85235ddd37ce93818c34f19857d08ea357b7c13a2c6ec5de87691ba54b949e"}
Sep 09 16:27:10.710576 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:10.710525    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5" podStartSLOduration=1.71050846 podStartE2EDuration="1.71050846s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 16:27:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 16:27:10.708702834 +0000 UTC m=+1106.126552846" watchObservedRunningTime="2026-09-09 16:27:10.71050846 +0000 UTC m=+1106.128358459"
Sep 09 16:27:11.567305 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:11.567265    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5"
Sep 09 16:27:11.572319 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:11.572293    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5"
Sep 09 16:27:11.695959 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:11.695928    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5"
Sep 09 16:27:11.696956 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:11.696939    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5"
Sep 09 16:27:13.708403 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:13.708370    2567 generic.go:356] "Generic (PLEG): container finished" podID="2b6f1200-474b-4101-b718-2b2d0cdd490a" containerID="7f1d5470d42639ebd84df7dd2a9044de67d9eaac7124a4f43e899b86c237895f" exitCode=0
Sep 09 16:27:13.708838 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:13.708438    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-743aed-openshift-default-658d874dc4-lzxsj" event={"ID":"2b6f1200-474b-4101-b718-2b2d0cdd490a","Type":"ContainerDied","Data":"7f1d5470d42639ebd84df7dd2a9044de67d9eaac7124a4f43e899b86c237895f"}
Sep 09 16:27:13.708838 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:13.708529    2567 scope.go:117] "RemoveContainer" containerID="5d6d5a000288ced75e228c00aa5c4477a894bb9748d4b724422803e7edfe39c4"
Sep 09 16:27:13.727862 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:27:13.727840    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d6d5a000288ced75e228c00aa5c4477a894bb9748d4b724422803e7edfe39c4\": container with ID starting with 5d6d5a000288ced75e228c00aa5c4477a894bb9748d4b724422803e7edfe39c4 not found: ID does not exist" containerID="5d6d5a000288ced75e228c00aa5c4477a894bb9748d4b724422803e7edfe39c4"
Sep 09 16:27:13.731328 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:13.731288    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-743aed-openshift-default-658d874dc4-lzxsj"
Sep 09 16:27:13.895224 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:13.895186    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b6f1200-474b-4101-b718-2b2d0cdd490a-istio-envoy\") pod \"2b6f1200-474b-4101-b718-2b2d0cdd490a\" (UID: \"2b6f1200-474b-4101-b718-2b2d0cdd490a\") "
Sep 09 16:27:13.895224 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:13.895228    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b6f1200-474b-4101-b718-2b2d0cdd490a-credential-socket\") pod \"2b6f1200-474b-4101-b718-2b2d0cdd490a\" (UID: \"2b6f1200-474b-4101-b718-2b2d0cdd490a\") "
Sep 09 16:27:13.895512 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:13.895249    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b6f1200-474b-4101-b718-2b2d0cdd490a-istio-podinfo\") pod \"2b6f1200-474b-4101-b718-2b2d0cdd490a\" (UID: \"2b6f1200-474b-4101-b718-2b2d0cdd490a\") "
Sep 09 16:27:13.895512 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:13.895342    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b6f1200-474b-4101-b718-2b2d0cdd490a-workload-socket\") pod \"2b6f1200-474b-4101-b718-2b2d0cdd490a\" (UID: \"2b6f1200-474b-4101-b718-2b2d0cdd490a\") "
Sep 09 16:27:13.895512 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:13.895366    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b6f1200-474b-4101-b718-2b2d0cdd490a-istio-data\") pod \"2b6f1200-474b-4101-b718-2b2d0cdd490a\" (UID: \"2b6f1200-474b-4101-b718-2b2d0cdd490a\") "
Sep 09 16:27:13.895512 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:13.895408    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b6f1200-474b-4101-b718-2b2d0cdd490a-istio-token\") pod \"2b6f1200-474b-4101-b718-2b2d0cdd490a\" (UID: \"2b6f1200-474b-4101-b718-2b2d0cdd490a\") "
Sep 09 16:27:13.895512 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:13.895431    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b6f1200-474b-4101-b718-2b2d0cdd490a-istiod-ca-cert\") pod \"2b6f1200-474b-4101-b718-2b2d0cdd490a\" (UID: \"2b6f1200-474b-4101-b718-2b2d0cdd490a\") "
Sep 09 16:27:13.895512 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:13.895488    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b6f1200-474b-4101-b718-2b2d0cdd490a-workload-certs\") pod \"2b6f1200-474b-4101-b718-2b2d0cdd490a\" (UID: \"2b6f1200-474b-4101-b718-2b2d0cdd490a\") "
Sep 09 16:27:13.895826 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:13.895526    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r2nd6\" (UniqueName: \"kubernetes.io/projected/2b6f1200-474b-4101-b718-2b2d0cdd490a-kube-api-access-r2nd6\") pod \"2b6f1200-474b-4101-b718-2b2d0cdd490a\" (UID: \"2b6f1200-474b-4101-b718-2b2d0cdd490a\") "
Sep 09 16:27:13.895826 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:13.895572    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b6f1200-474b-4101-b718-2b2d0cdd490a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2b6f1200-474b-4101-b718-2b2d0cdd490a" (UID: "2b6f1200-474b-4101-b718-2b2d0cdd490a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:27:13.895826 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:13.895727    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b6f1200-474b-4101-b718-2b2d0cdd490a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2b6f1200-474b-4101-b718-2b2d0cdd490a" (UID: "2b6f1200-474b-4101-b718-2b2d0cdd490a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:27:13.895826 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:13.895805    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b6f1200-474b-4101-b718-2b2d0cdd490a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2b6f1200-474b-4101-b718-2b2d0cdd490a" (UID: "2b6f1200-474b-4101-b718-2b2d0cdd490a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:27:13.896029 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:13.895825    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b6f1200-474b-4101-b718-2b2d0cdd490a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2b6f1200-474b-4101-b718-2b2d0cdd490a" (UID: "2b6f1200-474b-4101-b718-2b2d0cdd490a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 16:27:13.896029 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:13.895926    2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b6f1200-474b-4101-b718-2b2d0cdd490a-istiod-ca-cert\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:27:13.896029 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:13.895944    2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b6f1200-474b-4101-b718-2b2d0cdd490a-workload-certs\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:27:13.896029 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:13.895955    2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b6f1200-474b-4101-b718-2b2d0cdd490a-credential-socket\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:27:13.896029 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:13.895970    2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b6f1200-474b-4101-b718-2b2d0cdd490a-workload-socket\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:27:13.896449 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:13.896375    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b6f1200-474b-4101-b718-2b2d0cdd490a-istio-data" (OuterVolumeSpecName: "istio-data") pod "2b6f1200-474b-4101-b718-2b2d0cdd490a" (UID: "2b6f1200-474b-4101-b718-2b2d0cdd490a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:27:13.897800 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:13.897778    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b6f1200-474b-4101-b718-2b2d0cdd490a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2b6f1200-474b-4101-b718-2b2d0cdd490a" (UID: "2b6f1200-474b-4101-b718-2b2d0cdd490a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:27:13.897889 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:13.897800    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2b6f1200-474b-4101-b718-2b2d0cdd490a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2b6f1200-474b-4101-b718-2b2d0cdd490a" (UID: "2b6f1200-474b-4101-b718-2b2d0cdd490a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 16:27:13.898300 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:13.898269    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b6f1200-474b-4101-b718-2b2d0cdd490a-kube-api-access-r2nd6" (OuterVolumeSpecName: "kube-api-access-r2nd6") pod "2b6f1200-474b-4101-b718-2b2d0cdd490a" (UID: "2b6f1200-474b-4101-b718-2b2d0cdd490a"). InnerVolumeSpecName "kube-api-access-r2nd6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 16:27:13.898389 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:13.898269    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b6f1200-474b-4101-b718-2b2d0cdd490a-istio-token" (OuterVolumeSpecName: "istio-token") pod "2b6f1200-474b-4101-b718-2b2d0cdd490a" (UID: "2b6f1200-474b-4101-b718-2b2d0cdd490a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 16:27:13.997225 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:13.997126    2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b6f1200-474b-4101-b718-2b2d0cdd490a-istio-envoy\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:27:13.997225 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:13.997164    2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b6f1200-474b-4101-b718-2b2d0cdd490a-istio-podinfo\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:27:13.997225 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:13.997173    2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b6f1200-474b-4101-b718-2b2d0cdd490a-istio-data\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:27:13.997225 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:13.997181    2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b6f1200-474b-4101-b718-2b2d0cdd490a-istio-token\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:27:13.997225 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:13.997189    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r2nd6\" (UniqueName: \"kubernetes.io/projected/2b6f1200-474b-4101-b718-2b2d0cdd490a-kube-api-access-r2nd6\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:27:14.619062 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:14.619023    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:27:14.715964 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:14.715928    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-743aed-openshift-default-658d874dc4-lzxsj"
Sep 09 16:27:14.716392 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:14.715962    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-743aed-openshift-default-658d874dc4-lzxsj" event={"ID":"2b6f1200-474b-4101-b718-2b2d0cdd490a","Type":"ContainerDied","Data":"5ee81144b41495f00945f4fafac8c75144d9e6f9036fa9c0cf605de9e2f4d09c"}
Sep 09 16:27:14.716392 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:14.716018    2567 scope.go:117] "RemoveContainer" containerID="7f1d5470d42639ebd84df7dd2a9044de67d9eaac7124a4f43e899b86c237895f"
Sep 09 16:27:14.737930 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:14.737900    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-743aed-openshift-default-658d874dc4-lzxsj"]
Sep 09 16:27:14.741579 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:14.741558    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-autores-743aed-openshift-default-658d874dc4-lzxsj"]
Sep 09 16:27:15.037245 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:15.037164    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b6f1200-474b-4101-b718-2b2d0cdd490a" path="/var/lib/kubelet/pods/2b6f1200-474b-4101-b718-2b2d0cdd490a/volumes"
Sep 09 16:27:30.066040 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:30.065998    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2dc4f62b-67kcp"]
Sep 09 16:27:30.066520 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:30.066501    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1902f142-3560-4d43-94d0-f8041d1eb0b3" containerName="istio-proxy"
Sep 09 16:27:30.066520 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:30.066520    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="1902f142-3560-4d43-94d0-f8041d1eb0b3" containerName="istio-proxy"
Sep 09 16:27:30.066616 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:30.066529    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b6f1200-474b-4101-b718-2b2d0cdd490a" containerName="istio-proxy"
Sep 09 16:27:30.066616 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:30.066535    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b6f1200-474b-4101-b718-2b2d0cdd490a" containerName="istio-proxy"
Sep 09 16:27:30.066616 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:30.066553    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b6f1200-474b-4101-b718-2b2d0cdd490a" containerName="istio-proxy"
Sep 09 16:27:30.066616 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:30.066558    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b6f1200-474b-4101-b718-2b2d0cdd490a" containerName="istio-proxy"
Sep 09 16:27:30.066616 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:30.066615    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="1902f142-3560-4d43-94d0-f8041d1eb0b3" containerName="istio-proxy"
Sep 09 16:27:30.066798 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:30.066622    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b6f1200-474b-4101-b718-2b2d0cdd490a" containerName="istio-proxy"
Sep 09 16:27:30.066798 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:30.066629    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b6f1200-474b-4101-b718-2b2d0cdd490a" containerName="istio-proxy"
Sep 09 16:27:30.071111 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:30.071095    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2dc4f62b-67kcp"
Sep 09 16:27:30.073250 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:30.073223    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 09 16:27:30.073338 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:30.073250    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wxghs\""
Sep 09 16:27:30.077638 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:30.077613    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2dc4f62b-67kcp"]
Sep 09 16:27:30.150649 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:30.150609    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe352310-6a7e-4f50-872c-9fd45665af55-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-2dc4f62b-67kcp\" (UID: \"fe352310-6a7e-4f50-872c-9fd45665af55\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2dc4f62b-67kcp"
Sep 09 16:27:30.251296 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:30.251249    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe352310-6a7e-4f50-872c-9fd45665af55-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-2dc4f62b-67kcp\" (UID: \"fe352310-6a7e-4f50-872c-9fd45665af55\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2dc4f62b-67kcp"
Sep 09 16:27:30.251930 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:30.251910    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe352310-6a7e-4f50-872c-9fd45665af55-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-2dc4f62b-67kcp\" (UID: \"fe352310-6a7e-4f50-872c-9fd45665af55\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2dc4f62b-67kcp"
Sep 09 16:27:30.383708 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:30.383659    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2dc4f62b-67kcp"
Sep 09 16:27:30.652201 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:30.652067    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2dc4f62b-67kcp"]
Sep 09 16:27:30.656178 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:27:30.656149    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe352310_6a7e_4f50_872c_9fd45665af55.slice/crio-f2493eff1874053c2f3e9924c64687ff62b4e5d68065b91e43dae424a2874ae1 WatchSource:0}: Error finding container f2493eff1874053c2f3e9924c64687ff62b4e5d68065b91e43dae424a2874ae1: Status 404 returned error can't find the container with id f2493eff1874053c2f3e9924c64687ff62b4e5d68065b91e43dae424a2874ae1
Sep 09 16:27:30.807374 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:30.807329    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2dc4f62b-67kcp" event={"ID":"fe352310-6a7e-4f50-872c-9fd45665af55","Type":"ContainerStarted","Data":"f2493eff1874053c2f3e9924c64687ff62b4e5d68065b91e43dae424a2874ae1"}
Sep 09 16:27:31.816868 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:31.816824    2567 generic.go:356] "Generic (PLEG): container finished" podID="fe352310-6a7e-4f50-872c-9fd45665af55" containerID="b310bced0a26fd152d3cbbfb2ba1dc46540aaaa24bd6e31b3ac497cc9eef51de" exitCode=0
Sep 09 16:27:31.817261 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:31.816910    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2dc4f62b-67kcp" event={"ID":"fe352310-6a7e-4f50-872c-9fd45665af55","Type":"ContainerDied","Data":"b310bced0a26fd152d3cbbfb2ba1dc46540aaaa24bd6e31b3ac497cc9eef51de"}
Sep 09 16:27:32.973281 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:32.973255    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2dc4f62b-67kcp"
Sep 09 16:27:33.079983 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:33.079897    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe352310-6a7e-4f50-872c-9fd45665af55-maas-api-service-ca\") pod \"fe352310-6a7e-4f50-872c-9fd45665af55\" (UID: \"fe352310-6a7e-4f50-872c-9fd45665af55\") "
Sep 09 16:27:33.080283 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:33.080258    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe352310-6a7e-4f50-872c-9fd45665af55-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fe352310-6a7e-4f50-872c-9fd45665af55" (UID: "fe352310-6a7e-4f50-872c-9fd45665af55"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 16:27:33.181361 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:33.181320    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe352310-6a7e-4f50-872c-9fd45665af55-maas-api-service-ca\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:27:33.830378 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:33.830348    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2dc4f62b-67kcp"
Sep 09 16:27:33.830612 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:33.830377    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2dc4f62b-67kcp" event={"ID":"fe352310-6a7e-4f50-872c-9fd45665af55","Type":"ContainerDied","Data":"f2493eff1874053c2f3e9924c64687ff62b4e5d68065b91e43dae424a2874ae1"}
Sep 09 16:27:33.830612 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:33.830422    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f2493eff1874053c2f3e9924c64687ff62b4e5d68065b91e43dae424a2874ae1"
Sep 09 16:27:37.360029 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:37.359992    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2dc4f62b-67kcp"]
Sep 09 16:27:37.362606 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:37.362581    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2dc4f62b-67kcp"]
Sep 09 16:27:37.707588 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:37.707425    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:27:39.036109 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:39.036075    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe352310-6a7e-4f50-872c-9fd45665af55" path="/var/lib/kubelet/pods/fe352310-6a7e-4f50-872c-9fd45665af55/volumes"
Sep 09 16:27:39.861949 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:39.861913    2567 generic.go:356] "Generic (PLEG): container finished" podID="fc3a0139-4de3-4798-ab06-1ee71ff29ca3" containerID="f5b7d8608c786adb0f387e0949d7f66e07534a9c5785777a8e67d97f29c7caa1" exitCode=0
Sep 09 16:27:39.862199 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:39.861987    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5" event={"ID":"fc3a0139-4de3-4798-ab06-1ee71ff29ca3","Type":"ContainerDied","Data":"f5b7d8608c786adb0f387e0949d7f66e07534a9c5785777a8e67d97f29c7caa1"}
Sep 09 16:27:39.862427 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:39.862413    2567 scope.go:117] "RemoveContainer" containerID="f5b7d8608c786adb0f387e0949d7f66e07534a9c5785777a8e67d97f29c7caa1"
Sep 09 16:27:39.863576 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:39.863542    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:27:39.863676 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:39.863616    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:27:39.863676 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:39.863647    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:27:40.869026 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:40.868990    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5" event={"ID":"fc3a0139-4de3-4798-ab06-1ee71ff29ca3","Type":"ContainerStarted","Data":"5a7cfb11415dd73407918a86adea3c39ee6ec4939bc7d81a41324b251500c9a2"}
Sep 09 16:27:41.567215 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:41.567181    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5"
Sep 09 16:27:41.568418 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:41.568390    2567 patch_prober.go:28] interesting pod/e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Sep 09 16:27:41.568576 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:41.568447    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5" podUID="fc3a0139-4de3-4798-ab06-1ee71ff29ca3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Sep 09 16:27:41.696218 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:41.696172    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5"
Sep 09 16:27:42.566801 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:42.566767    2567 patch_prober.go:28] interesting pod/e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Sep 09 16:27:42.567186 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:42.566824    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5" podUID="fc3a0139-4de3-4798-ab06-1ee71ff29ca3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Sep 09 16:27:43.566788 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:43.566756    2567 patch_prober.go:28] interesting pod/e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Sep 09 16:27:43.566955 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:43.566823    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5" podUID="fc3a0139-4de3-4798-ab06-1ee71ff29ca3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Sep 09 16:27:44.567552 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:44.567520    2567 patch_prober.go:28] interesting pod/e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Sep 09 16:27:44.567941 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:44.567577    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5" podUID="fc3a0139-4de3-4798-ab06-1ee71ff29ca3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Sep 09 16:27:45.566989 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:45.566950    2567 patch_prober.go:28] interesting pod/e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Sep 09 16:27:45.567171 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:45.567026    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5" podUID="fc3a0139-4de3-4798-ab06-1ee71ff29ca3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Sep 09 16:27:46.571532 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:46.571496    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5"
Sep 09 16:27:46.572724 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:46.572705    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5"
Sep 09 16:27:51.815044 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:27:51.815001    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:28:00.553491 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:00.553436    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3ad4f1-wwh4l"]
Sep 09 16:28:00.553967 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:00.553812    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe352310-6a7e-4f50-872c-9fd45665af55" containerName="revoke-keys"
Sep 09 16:28:00.553967 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:00.553823    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe352310-6a7e-4f50-872c-9fd45665af55" containerName="revoke-keys"
Sep 09 16:28:00.553967 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:00.553896    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe352310-6a7e-4f50-872c-9fd45665af55" containerName="revoke-keys"
Sep 09 16:28:00.558246 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:00.558229    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3ad4f1-wwh4l"
Sep 09 16:28:00.560788 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:00.560762    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 09 16:28:00.560924 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:00.560815    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wxghs\""
Sep 09 16:28:00.567241 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:00.567214    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3ad4f1-wwh4l"]
Sep 09 16:28:00.667711 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:00.667680    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/08cf846c-403f-4fcb-af56-ca960fe4d55c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-3ad4f1-wwh4l\" (UID: \"08cf846c-403f-4fcb-af56-ca960fe4d55c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3ad4f1-wwh4l"
Sep 09 16:28:00.768959 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:00.768921    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/08cf846c-403f-4fcb-af56-ca960fe4d55c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-3ad4f1-wwh4l\" (UID: \"08cf846c-403f-4fcb-af56-ca960fe4d55c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3ad4f1-wwh4l"
Sep 09 16:28:00.769605 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:00.769585    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/08cf846c-403f-4fcb-af56-ca960fe4d55c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-3ad4f1-wwh4l\" (UID: \"08cf846c-403f-4fcb-af56-ca960fe4d55c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3ad4f1-wwh4l"
Sep 09 16:28:00.869968 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:00.869930    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3ad4f1-wwh4l"
Sep 09 16:28:01.161751 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:01.161676    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3ad4f1-wwh4l"]
Sep 09 16:28:01.165428 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:28:01.165399    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08cf846c_403f_4fcb_af56_ca960fe4d55c.slice/crio-4a2ea52dced32c8897321426f95990e0c7104d0cc051b1f0515f8191937c4d16 WatchSource:0}: Error finding container 4a2ea52dced32c8897321426f95990e0c7104d0cc051b1f0515f8191937c4d16: Status 404 returned error can't find the container with id 4a2ea52dced32c8897321426f95990e0c7104d0cc051b1f0515f8191937c4d16
Sep 09 16:28:01.981334 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:01.981296    2567 generic.go:356] "Generic (PLEG): container finished" podID="08cf846c-403f-4fcb-af56-ca960fe4d55c" containerID="06efe5d24781058a08f46b2ac5a940a008d69b3e6acb8518a92b560528e60dac" exitCode=0
Sep 09 16:28:01.981813 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:01.981381    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3ad4f1-wwh4l" event={"ID":"08cf846c-403f-4fcb-af56-ca960fe4d55c","Type":"ContainerDied","Data":"06efe5d24781058a08f46b2ac5a940a008d69b3e6acb8518a92b560528e60dac"}
Sep 09 16:28:01.981813 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:01.981493    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3ad4f1-wwh4l" event={"ID":"08cf846c-403f-4fcb-af56-ca960fe4d55c","Type":"ContainerStarted","Data":"4a2ea52dced32c8897321426f95990e0c7104d0cc051b1f0515f8191937c4d16"}
Sep 09 16:28:03.119560 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:03.119537    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3ad4f1-wwh4l"
Sep 09 16:28:03.194877 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:03.194839    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/08cf846c-403f-4fcb-af56-ca960fe4d55c-maas-api-service-ca\") pod \"08cf846c-403f-4fcb-af56-ca960fe4d55c\" (UID: \"08cf846c-403f-4fcb-af56-ca960fe4d55c\") "
Sep 09 16:28:03.195260 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:03.195232    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08cf846c-403f-4fcb-af56-ca960fe4d55c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "08cf846c-403f-4fcb-af56-ca960fe4d55c" (UID: "08cf846c-403f-4fcb-af56-ca960fe4d55c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 16:28:03.296394 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:03.296298    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/08cf846c-403f-4fcb-af56-ca960fe4d55c-maas-api-service-ca\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:28:03.993275 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:03.993246    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3ad4f1-wwh4l"
Sep 09 16:28:03.993441 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:03.993292    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3ad4f1-wwh4l" event={"ID":"08cf846c-403f-4fcb-af56-ca960fe4d55c","Type":"ContainerDied","Data":"4a2ea52dced32c8897321426f95990e0c7104d0cc051b1f0515f8191937c4d16"}
Sep 09 16:28:03.993441 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:03.993339    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4a2ea52dced32c8897321426f95990e0c7104d0cc051b1f0515f8191937c4d16"
Sep 09 16:28:10.609821 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:10.609734    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:28:11.205054 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:11.204993    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3ad4f1-wwh4l"]
Sep 09 16:28:11.207183 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:11.207162    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3ad4f1-wwh4l"]
Sep 09 16:28:13.036099 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:13.036066    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="08cf846c-403f-4fcb-af56-ca960fe4d55c" path="/var/lib/kubelet/pods/08cf846c-403f-4fcb-af56-ca960fe4d55c/volumes"
Sep 09 16:28:17.733017 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:17.732985    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5"]
Sep 09 16:28:17.733449 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:17.733198    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5" podUID="fc3a0139-4de3-4798-ab06-1ee71ff29ca3" containerName="istio-proxy" containerID="cri-o://5a7cfb11415dd73407918a86adea3c39ee6ec4939bc7d81a41324b251500c9a2" gracePeriod=30
Sep 09 16:28:22.985038 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:22.985009    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5"
Sep 09 16:28:23.099335 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:23.099305    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc3a0139-4de3-4798-ab06-1ee71ff29ca3-istio-envoy\") pod \"fc3a0139-4de3-4798-ab06-1ee71ff29ca3\" (UID: \"fc3a0139-4de3-4798-ab06-1ee71ff29ca3\") "
Sep 09 16:28:23.099335 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:23.099339    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc3a0139-4de3-4798-ab06-1ee71ff29ca3-istiod-ca-cert\") pod \"fc3a0139-4de3-4798-ab06-1ee71ff29ca3\" (UID: \"fc3a0139-4de3-4798-ab06-1ee71ff29ca3\") "
Sep 09 16:28:23.099621 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:23.099365    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc3a0139-4de3-4798-ab06-1ee71ff29ca3-istio-data\") pod \"fc3a0139-4de3-4798-ab06-1ee71ff29ca3\" (UID: \"fc3a0139-4de3-4798-ab06-1ee71ff29ca3\") "
Sep 09 16:28:23.099621 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:23.099411    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc3a0139-4de3-4798-ab06-1ee71ff29ca3-istio-token\") pod \"fc3a0139-4de3-4798-ab06-1ee71ff29ca3\" (UID: \"fc3a0139-4de3-4798-ab06-1ee71ff29ca3\") "
Sep 09 16:28:23.099621 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:23.099492    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc3a0139-4de3-4798-ab06-1ee71ff29ca3-credential-socket\") pod \"fc3a0139-4de3-4798-ab06-1ee71ff29ca3\" (UID: \"fc3a0139-4de3-4798-ab06-1ee71ff29ca3\") "
Sep 09 16:28:23.099621 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:23.099543    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc3a0139-4de3-4798-ab06-1ee71ff29ca3-istio-podinfo\") pod \"fc3a0139-4de3-4798-ab06-1ee71ff29ca3\" (UID: \"fc3a0139-4de3-4798-ab06-1ee71ff29ca3\") "
Sep 09 16:28:23.099621 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:23.099596    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc3a0139-4de3-4798-ab06-1ee71ff29ca3-workload-socket\") pod \"fc3a0139-4de3-4798-ab06-1ee71ff29ca3\" (UID: \"fc3a0139-4de3-4798-ab06-1ee71ff29ca3\") "
Sep 09 16:28:23.099891 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:23.099631    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6pdlx\" (UniqueName: \"kubernetes.io/projected/fc3a0139-4de3-4798-ab06-1ee71ff29ca3-kube-api-access-6pdlx\") pod \"fc3a0139-4de3-4798-ab06-1ee71ff29ca3\" (UID: \"fc3a0139-4de3-4798-ab06-1ee71ff29ca3\") "
Sep 09 16:28:23.099891 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:23.099656    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc3a0139-4de3-4798-ab06-1ee71ff29ca3-workload-certs\") pod \"fc3a0139-4de3-4798-ab06-1ee71ff29ca3\" (UID: \"fc3a0139-4de3-4798-ab06-1ee71ff29ca3\") "
Sep 09 16:28:23.099891 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:23.099814    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc3a0139-4de3-4798-ab06-1ee71ff29ca3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fc3a0139-4de3-4798-ab06-1ee71ff29ca3" (UID: "fc3a0139-4de3-4798-ab06-1ee71ff29ca3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:28:23.099891 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:23.099829    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc3a0139-4de3-4798-ab06-1ee71ff29ca3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fc3a0139-4de3-4798-ab06-1ee71ff29ca3" (UID: "fc3a0139-4de3-4798-ab06-1ee71ff29ca3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 16:28:23.100105 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:23.099949    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc3a0139-4de3-4798-ab06-1ee71ff29ca3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fc3a0139-4de3-4798-ab06-1ee71ff29ca3" (UID: "fc3a0139-4de3-4798-ab06-1ee71ff29ca3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:28:23.100105 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:23.100083    2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc3a0139-4de3-4798-ab06-1ee71ff29ca3-istiod-ca-cert\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:28:23.100222 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:23.100104    2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc3a0139-4de3-4798-ab06-1ee71ff29ca3-credential-socket\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:28:23.100222 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:23.100120    2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc3a0139-4de3-4798-ab06-1ee71ff29ca3-workload-socket\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:28:23.100364 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:23.100339    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc3a0139-4de3-4798-ab06-1ee71ff29ca3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fc3a0139-4de3-4798-ab06-1ee71ff29ca3" (UID: "fc3a0139-4de3-4798-ab06-1ee71ff29ca3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:28:23.100471 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:23.100384    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc3a0139-4de3-4798-ab06-1ee71ff29ca3-istio-data" (OuterVolumeSpecName: "istio-data") pod "fc3a0139-4de3-4798-ab06-1ee71ff29ca3" (UID: "fc3a0139-4de3-4798-ab06-1ee71ff29ca3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:28:23.102014 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:23.101992    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc3a0139-4de3-4798-ab06-1ee71ff29ca3-istio-token" (OuterVolumeSpecName: "istio-token") pod "fc3a0139-4de3-4798-ab06-1ee71ff29ca3" (UID: "fc3a0139-4de3-4798-ab06-1ee71ff29ca3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 16:28:23.102260 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:23.102237    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc3a0139-4de3-4798-ab06-1ee71ff29ca3-kube-api-access-6pdlx" (OuterVolumeSpecName: "kube-api-access-6pdlx") pod "fc3a0139-4de3-4798-ab06-1ee71ff29ca3" (UID: "fc3a0139-4de3-4798-ab06-1ee71ff29ca3"). InnerVolumeSpecName "kube-api-access-6pdlx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 16:28:23.102426 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:23.102404    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc3a0139-4de3-4798-ab06-1ee71ff29ca3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fc3a0139-4de3-4798-ab06-1ee71ff29ca3" (UID: "fc3a0139-4de3-4798-ab06-1ee71ff29ca3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:28:23.102543 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:23.102521    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fc3a0139-4de3-4798-ab06-1ee71ff29ca3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fc3a0139-4de3-4798-ab06-1ee71ff29ca3" (UID: "fc3a0139-4de3-4798-ab06-1ee71ff29ca3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 16:28:23.103698 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:23.103674    2567 generic.go:356] "Generic (PLEG): container finished" podID="fc3a0139-4de3-4798-ab06-1ee71ff29ca3" containerID="5a7cfb11415dd73407918a86adea3c39ee6ec4939bc7d81a41324b251500c9a2" exitCode=0
Sep 09 16:28:23.103786 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:23.103743    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5"
Sep 09 16:28:23.103786 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:23.103754    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5" event={"ID":"fc3a0139-4de3-4798-ab06-1ee71ff29ca3","Type":"ContainerDied","Data":"5a7cfb11415dd73407918a86adea3c39ee6ec4939bc7d81a41324b251500c9a2"}
Sep 09 16:28:23.103892 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:23.103807    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5" event={"ID":"fc3a0139-4de3-4798-ab06-1ee71ff29ca3","Type":"ContainerDied","Data":"fd85235ddd37ce93818c34f19857d08ea357b7c13a2c6ec5de87691ba54b949e"}
Sep 09 16:28:23.103892 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:23.103831    2567 scope.go:117] "RemoveContainer" containerID="5a7cfb11415dd73407918a86adea3c39ee6ec4939bc7d81a41324b251500c9a2"
Sep 09 16:28:23.123764 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:23.123741    2567 scope.go:117] "RemoveContainer" containerID="f5b7d8608c786adb0f387e0949d7f66e07534a9c5785777a8e67d97f29c7caa1"
Sep 09 16:28:23.132021 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:23.131999    2567 scope.go:117] "RemoveContainer" containerID="5a7cfb11415dd73407918a86adea3c39ee6ec4939bc7d81a41324b251500c9a2"
Sep 09 16:28:23.132302 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:28:23.132283    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a7cfb11415dd73407918a86adea3c39ee6ec4939bc7d81a41324b251500c9a2\": container with ID starting with 5a7cfb11415dd73407918a86adea3c39ee6ec4939bc7d81a41324b251500c9a2 not found: ID does not exist" containerID="5a7cfb11415dd73407918a86adea3c39ee6ec4939bc7d81a41324b251500c9a2"
Sep 09 16:28:23.132363 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:23.132309    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a7cfb11415dd73407918a86adea3c39ee6ec4939bc7d81a41324b251500c9a2"} err="failed to get container status \"5a7cfb11415dd73407918a86adea3c39ee6ec4939bc7d81a41324b251500c9a2\": rpc error: code = NotFound desc = could not find container \"5a7cfb11415dd73407918a86adea3c39ee6ec4939bc7d81a41324b251500c9a2\": container with ID starting with 5a7cfb11415dd73407918a86adea3c39ee6ec4939bc7d81a41324b251500c9a2 not found: ID does not exist"
Sep 09 16:28:23.132363 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:23.132327    2567 scope.go:117] "RemoveContainer" containerID="f5b7d8608c786adb0f387e0949d7f66e07534a9c5785777a8e67d97f29c7caa1"
Sep 09 16:28:23.132600 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:28:23.132576    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5b7d8608c786adb0f387e0949d7f66e07534a9c5785777a8e67d97f29c7caa1\": container with ID starting with f5b7d8608c786adb0f387e0949d7f66e07534a9c5785777a8e67d97f29c7caa1 not found: ID does not exist" containerID="f5b7d8608c786adb0f387e0949d7f66e07534a9c5785777a8e67d97f29c7caa1"
Sep 09 16:28:23.132761 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:23.132607    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5b7d8608c786adb0f387e0949d7f66e07534a9c5785777a8e67d97f29c7caa1"} err="failed to get container status \"f5b7d8608c786adb0f387e0949d7f66e07534a9c5785777a8e67d97f29c7caa1\": rpc error: code = NotFound desc = could not find container \"f5b7d8608c786adb0f387e0949d7f66e07534a9c5785777a8e67d97f29c7caa1\": container with ID starting with f5b7d8608c786adb0f387e0949d7f66e07534a9c5785777a8e67d97f29c7caa1 not found: ID does not exist"
Sep 09 16:28:23.135175 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:23.135150    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5"]
Sep 09 16:28:23.140234 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:23.140208    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-explicit-3ad4f1-openshift-default-d5bc8866d-89gh5"]
Sep 09 16:28:23.200694 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:23.200661    2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc3a0139-4de3-4798-ab06-1ee71ff29ca3-istio-envoy\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:28:23.200694 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:23.200689    2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc3a0139-4de3-4798-ab06-1ee71ff29ca3-istio-data\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:28:23.200694 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:23.200697    2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc3a0139-4de3-4798-ab06-1ee71ff29ca3-istio-token\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:28:23.200694 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:23.200704    2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc3a0139-4de3-4798-ab06-1ee71ff29ca3-istio-podinfo\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:28:23.200944 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:23.200713    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6pdlx\" (UniqueName: \"kubernetes.io/projected/fc3a0139-4de3-4798-ab06-1ee71ff29ca3-kube-api-access-6pdlx\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:28:23.200944 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:23.200721    2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc3a0139-4de3-4798-ab06-1ee71ff29ca3-workload-certs\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:28:23.805425 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:23.805391    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:28:25.035547 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:25.035517    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc3a0139-4de3-4798-ab06-1ee71ff29ca3" path="/var/lib/kubelet/pods/fc3a0139-4de3-4798-ab06-1ee71ff29ca3/volumes"
Sep 09 16:28:34.019998 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:34.019966    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:28:45.111343 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:45.111314    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-xwlzs_16ecfb4d-7944-465e-a38e-d8c5bda6b1ac/console-operator/2.log"
Sep 09 16:28:45.117863 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:45.117839    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-xwlzs_16ecfb4d-7944-465e-a38e-d8c5bda6b1ac/console-operator/2.log"
Sep 09 16:28:48.717726 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:48.717686    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:28:51.746353 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:51.746312    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-6d44b6/test-model-6d44b6-kserve-67d6d975d9-xw82r"]
Sep 09 16:28:52.835209 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:52.835165    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6d44b6-ghmgp"]
Sep 09 16:28:52.835753 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:52.835731    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc3a0139-4de3-4798-ab06-1ee71ff29ca3" containerName="istio-proxy"
Sep 09 16:28:52.835753 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:52.835755    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc3a0139-4de3-4798-ab06-1ee71ff29ca3" containerName="istio-proxy"
Sep 09 16:28:52.835895 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:52.835772    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc3a0139-4de3-4798-ab06-1ee71ff29ca3" containerName="istio-proxy"
Sep 09 16:28:52.835895 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:52.835782    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc3a0139-4de3-4798-ab06-1ee71ff29ca3" containerName="istio-proxy"
Sep 09 16:28:52.835895 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:52.835820    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08cf846c-403f-4fcb-af56-ca960fe4d55c" containerName="revoke-keys"
Sep 09 16:28:52.835895 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:52.835829    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="08cf846c-403f-4fcb-af56-ca960fe4d55c" containerName="revoke-keys"
Sep 09 16:28:52.836089 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:52.835920    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc3a0139-4de3-4798-ab06-1ee71ff29ca3" containerName="istio-proxy"
Sep 09 16:28:52.836089 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:52.835931    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="08cf846c-403f-4fcb-af56-ca960fe4d55c" containerName="revoke-keys"
Sep 09 16:28:52.840310 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:52.840279    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6d44b6-ghmgp"
Sep 09 16:28:52.842989 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:52.842783    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wxghs\""
Sep 09 16:28:52.842989 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:52.842913    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 09 16:28:52.845604 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:52.845579    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6d44b6-ghmgp"]
Sep 09 16:28:52.970566 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:52.970537    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ac98ebf-431d-4c02-ba73-b3ce358742fc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-6d44b6-ghmgp\" (UID: \"1ac98ebf-431d-4c02-ba73-b3ce358742fc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6d44b6-ghmgp"
Sep 09 16:28:53.071891 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:53.071853    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ac98ebf-431d-4c02-ba73-b3ce358742fc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-6d44b6-ghmgp\" (UID: \"1ac98ebf-431d-4c02-ba73-b3ce358742fc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6d44b6-ghmgp"
Sep 09 16:28:53.072600 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:53.072577    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ac98ebf-431d-4c02-ba73-b3ce358742fc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-6d44b6-ghmgp\" (UID: \"1ac98ebf-431d-4c02-ba73-b3ce358742fc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6d44b6-ghmgp"
Sep 09 16:28:53.153831 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:53.153743    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6d44b6-ghmgp"
Sep 09 16:28:53.423756 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:53.423730    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6d44b6-ghmgp"]
Sep 09 16:28:53.426021 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:28:53.425994    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ac98ebf_431d_4c02_ba73_b3ce358742fc.slice/crio-6f67b3cb47541bca96ee18c0065268363985948c5865b3e7dbb9e85e3679cf8b WatchSource:0}: Error finding container 6f67b3cb47541bca96ee18c0065268363985948c5865b3e7dbb9e85e3679cf8b: Status 404 returned error can't find the container with id 6f67b3cb47541bca96ee18c0065268363985948c5865b3e7dbb9e85e3679cf8b
Sep 09 16:28:54.259317 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:54.259220    2567 generic.go:356] "Generic (PLEG): container finished" podID="1ac98ebf-431d-4c02-ba73-b3ce358742fc" containerID="528f4d915cf01263331bfbabb99a58410e059bb8f5d1bf8457abb5aff08ded26" exitCode=0
Sep 09 16:28:54.259317 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:54.259256    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6d44b6-ghmgp" event={"ID":"1ac98ebf-431d-4c02-ba73-b3ce358742fc","Type":"ContainerDied","Data":"528f4d915cf01263331bfbabb99a58410e059bb8f5d1bf8457abb5aff08ded26"}
Sep 09 16:28:54.259745 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:54.259320    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6d44b6-ghmgp" event={"ID":"1ac98ebf-431d-4c02-ba73-b3ce358742fc","Type":"ContainerStarted","Data":"6f67b3cb47541bca96ee18c0065268363985948c5865b3e7dbb9e85e3679cf8b"}
Sep 09 16:28:55.398692 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:55.398666    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6d44b6-ghmgp"
Sep 09 16:28:55.495342 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:55.495309    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ac98ebf-431d-4c02-ba73-b3ce358742fc-maas-api-service-ca\") pod \"1ac98ebf-431d-4c02-ba73-b3ce358742fc\" (UID: \"1ac98ebf-431d-4c02-ba73-b3ce358742fc\") "
Sep 09 16:28:55.495690 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:55.495665    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ac98ebf-431d-4c02-ba73-b3ce358742fc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1ac98ebf-431d-4c02-ba73-b3ce358742fc" (UID: "1ac98ebf-431d-4c02-ba73-b3ce358742fc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 16:28:55.596717 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:55.596686    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ac98ebf-431d-4c02-ba73-b3ce358742fc-maas-api-service-ca\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:28:56.270249 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:56.270214    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6d44b6-ghmgp"
Sep 09 16:28:56.270526 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:56.270239    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6d44b6-ghmgp" event={"ID":"1ac98ebf-431d-4c02-ba73-b3ce358742fc","Type":"ContainerDied","Data":"6f67b3cb47541bca96ee18c0065268363985948c5865b3e7dbb9e85e3679cf8b"}
Sep 09 16:28:56.270526 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:56.270332    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6f67b3cb47541bca96ee18c0065268363985948c5865b3e7dbb9e85e3679cf8b"
Sep 09 16:28:58.719281 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:28:58.719245    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:29:03.568921 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:03.568882    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6d44b6-ghmgp"]
Sep 09 16:29:03.571319 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:03.571293    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6d44b6-ghmgp"]
Sep 09 16:29:05.035840 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:05.035806    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ac98ebf-431d-4c02-ba73-b3ce358742fc" path="/var/lib/kubelet/pods/1ac98ebf-431d-4c02-ba73-b3ce358742fc/volumes"
Sep 09 16:29:06.766919 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:06.766845    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-6d44b6/test-model-6d44b6-kserve-67d6d975d9-xw82r" podUID="dd73e10d-18bb-42fd-be79-ae6949f19f36" containerName="main" containerID="cri-o://290736c36ea54c4a4ec869a74e67158bc387e4d6f9480168f098853c662705d6" gracePeriod=45
Sep 09 16:29:07.019667 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:07.019602    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-6d44b6/test-model-6d44b6-kserve-67d6d975d9-xw82r"
Sep 09 16:29:07.112112 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:07.112077    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd73e10d-18bb-42fd-be79-ae6949f19f36-dshm\") pod \"dd73e10d-18bb-42fd-be79-ae6949f19f36\" (UID: \"dd73e10d-18bb-42fd-be79-ae6949f19f36\") "
Sep 09 16:29:07.112305 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:07.112172    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd73e10d-18bb-42fd-be79-ae6949f19f36-tls-certs\") pod \"dd73e10d-18bb-42fd-be79-ae6949f19f36\" (UID: \"dd73e10d-18bb-42fd-be79-ae6949f19f36\") "
Sep 09 16:29:07.112305 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:07.112197    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd73e10d-18bb-42fd-be79-ae6949f19f36-home\") pod \"dd73e10d-18bb-42fd-be79-ae6949f19f36\" (UID: \"dd73e10d-18bb-42fd-be79-ae6949f19f36\") "
Sep 09 16:29:07.112305 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:07.112238    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xgmfb\" (UniqueName: \"kubernetes.io/projected/dd73e10d-18bb-42fd-be79-ae6949f19f36-kube-api-access-xgmfb\") pod \"dd73e10d-18bb-42fd-be79-ae6949f19f36\" (UID: \"dd73e10d-18bb-42fd-be79-ae6949f19f36\") "
Sep 09 16:29:07.112305 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:07.112283    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd73e10d-18bb-42fd-be79-ae6949f19f36-model-cache\") pod \"dd73e10d-18bb-42fd-be79-ae6949f19f36\" (UID: \"dd73e10d-18bb-42fd-be79-ae6949f19f36\") "
Sep 09 16:29:07.112548 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:07.112336    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd73e10d-18bb-42fd-be79-ae6949f19f36-tmp-dir\") pod \"dd73e10d-18bb-42fd-be79-ae6949f19f36\" (UID: \"dd73e10d-18bb-42fd-be79-ae6949f19f36\") "
Sep 09 16:29:07.112548 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:07.112505    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd73e10d-18bb-42fd-be79-ae6949f19f36-home" (OuterVolumeSpecName: "home") pod "dd73e10d-18bb-42fd-be79-ae6949f19f36" (UID: "dd73e10d-18bb-42fd-be79-ae6949f19f36"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:29:07.112858 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:07.112650    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd73e10d-18bb-42fd-be79-ae6949f19f36-model-cache" (OuterVolumeSpecName: "model-cache") pod "dd73e10d-18bb-42fd-be79-ae6949f19f36" (UID: "dd73e10d-18bb-42fd-be79-ae6949f19f36"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:29:07.112858 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:07.112713    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd73e10d-18bb-42fd-be79-ae6949f19f36-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "dd73e10d-18bb-42fd-be79-ae6949f19f36" (UID: "dd73e10d-18bb-42fd-be79-ae6949f19f36"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:29:07.112858 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:07.112803    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd73e10d-18bb-42fd-be79-ae6949f19f36-model-cache\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:29:07.112858 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:07.112822    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd73e10d-18bb-42fd-be79-ae6949f19f36-home\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:29:07.114660 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:07.114635    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd73e10d-18bb-42fd-be79-ae6949f19f36-dshm" (OuterVolumeSpecName: "dshm") pod "dd73e10d-18bb-42fd-be79-ae6949f19f36" (UID: "dd73e10d-18bb-42fd-be79-ae6949f19f36"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:29:07.114768 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:07.114654    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd73e10d-18bb-42fd-be79-ae6949f19f36-kube-api-access-xgmfb" (OuterVolumeSpecName: "kube-api-access-xgmfb") pod "dd73e10d-18bb-42fd-be79-ae6949f19f36" (UID: "dd73e10d-18bb-42fd-be79-ae6949f19f36"). InnerVolumeSpecName "kube-api-access-xgmfb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 16:29:07.114768 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:07.114689    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dd73e10d-18bb-42fd-be79-ae6949f19f36-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "dd73e10d-18bb-42fd-be79-ae6949f19f36" (UID: "dd73e10d-18bb-42fd-be79-ae6949f19f36"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 16:29:07.214244 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:07.214204    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xgmfb\" (UniqueName: \"kubernetes.io/projected/dd73e10d-18bb-42fd-be79-ae6949f19f36-kube-api-access-xgmfb\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:29:07.214244 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:07.214242    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd73e10d-18bb-42fd-be79-ae6949f19f36-tmp-dir\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:29:07.214244 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:07.214252    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd73e10d-18bb-42fd-be79-ae6949f19f36-dshm\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:29:07.214534 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:07.214259    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd73e10d-18bb-42fd-be79-ae6949f19f36-tls-certs\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:29:07.328745 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:07.328712    2567 generic.go:356] "Generic (PLEG): container finished" podID="dd73e10d-18bb-42fd-be79-ae6949f19f36" containerID="290736c36ea54c4a4ec869a74e67158bc387e4d6f9480168f098853c662705d6" exitCode=0
Sep 09 16:29:07.328912 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:07.328762    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-6d44b6/test-model-6d44b6-kserve-67d6d975d9-xw82r"
Sep 09 16:29:07.328912 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:07.328804    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-6d44b6/test-model-6d44b6-kserve-67d6d975d9-xw82r" event={"ID":"dd73e10d-18bb-42fd-be79-ae6949f19f36","Type":"ContainerDied","Data":"290736c36ea54c4a4ec869a74e67158bc387e4d6f9480168f098853c662705d6"}
Sep 09 16:29:07.328912 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:07.328852    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-6d44b6/test-model-6d44b6-kserve-67d6d975d9-xw82r" event={"ID":"dd73e10d-18bb-42fd-be79-ae6949f19f36","Type":"ContainerDied","Data":"749190ebe038a434b9a1f39aa55e3c89a04380b349d6fbbbeb5bcb34acd9abe7"}
Sep 09 16:29:07.328912 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:07.328870    2567 scope.go:117] "RemoveContainer" containerID="290736c36ea54c4a4ec869a74e67158bc387e4d6f9480168f098853c662705d6"
Sep 09 16:29:07.338509 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:07.338492    2567 scope.go:117] "RemoveContainer" containerID="290736c36ea54c4a4ec869a74e67158bc387e4d6f9480168f098853c662705d6"
Sep 09 16:29:07.338802 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:29:07.338783    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"290736c36ea54c4a4ec869a74e67158bc387e4d6f9480168f098853c662705d6\": container with ID starting with 290736c36ea54c4a4ec869a74e67158bc387e4d6f9480168f098853c662705d6 not found: ID does not exist" containerID="290736c36ea54c4a4ec869a74e67158bc387e4d6f9480168f098853c662705d6"
Sep 09 16:29:07.338870 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:07.338808    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"290736c36ea54c4a4ec869a74e67158bc387e4d6f9480168f098853c662705d6"} err="failed to get container status \"290736c36ea54c4a4ec869a74e67158bc387e4d6f9480168f098853c662705d6\": rpc error: code = NotFound desc = could not find container \"290736c36ea54c4a4ec869a74e67158bc387e4d6f9480168f098853c662705d6\": container with ID starting with 290736c36ea54c4a4ec869a74e67158bc387e4d6f9480168f098853c662705d6 not found: ID does not exist"
Sep 09 16:29:07.349224 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:07.349200    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-6d44b6/test-model-6d44b6-kserve-67d6d975d9-xw82r"]
Sep 09 16:29:07.350829 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:07.350810    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-6d44b6/test-model-6d44b6-kserve-67d6d975d9-xw82r"]
Sep 09 16:29:09.036397 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:09.036363    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd73e10d-18bb-42fd-be79-ae6949f19f36" path="/var/lib/kubelet/pods/dd73e10d-18bb-42fd-be79-ae6949f19f36/volumes"
Sep 09 16:29:09.612920 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:09.612881    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:29:09.909806 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:09.909698    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl"]
Sep 09 16:29:09.910325 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:09.910252    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl" podUID="f1e36da0-3a7c-4765-aeb6-0b6339a09cc4" containerName="istio-proxy" containerID="cri-o://75a99bdfb20bee38d1ed434e26628356d776f38035872feefd7b50f587a07fe1" gracePeriod=30
Sep 09 16:29:10.282378 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:10.282295    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c23332-z945b"]
Sep 09 16:29:10.282789 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:10.282680    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd73e10d-18bb-42fd-be79-ae6949f19f36" containerName="main"
Sep 09 16:29:10.282789 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:10.282692    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd73e10d-18bb-42fd-be79-ae6949f19f36" containerName="main"
Sep 09 16:29:10.282789 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:10.282701    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ac98ebf-431d-4c02-ba73-b3ce358742fc" containerName="revoke-keys"
Sep 09 16:29:10.282789 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:10.282706    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ac98ebf-431d-4c02-ba73-b3ce358742fc" containerName="revoke-keys"
Sep 09 16:29:10.282789 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:10.282762    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc3a0139-4de3-4798-ab06-1ee71ff29ca3" containerName="istio-proxy"
Sep 09 16:29:10.282789 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:10.282772    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd73e10d-18bb-42fd-be79-ae6949f19f36" containerName="main"
Sep 09 16:29:10.282789 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:10.282780    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ac98ebf-431d-4c02-ba73-b3ce358742fc" containerName="revoke-keys"
Sep 09 16:29:10.286985 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:10.286963    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c23332-z945b"
Sep 09 16:29:10.289173 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:10.289151    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 09 16:29:10.289309 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:10.289193    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wxghs\""
Sep 09 16:29:10.293645 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:10.293622    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c23332-z945b"]
Sep 09 16:29:10.445416 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:10.445378    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed0d0755-723c-446f-86fc-792619847d55-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-c23332-z945b\" (UID: \"ed0d0755-723c-446f-86fc-792619847d55\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c23332-z945b"
Sep 09 16:29:10.546861 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:10.546770    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed0d0755-723c-446f-86fc-792619847d55-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-c23332-z945b\" (UID: \"ed0d0755-723c-446f-86fc-792619847d55\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c23332-z945b"
Sep 09 16:29:10.547422 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:10.547400    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed0d0755-723c-446f-86fc-792619847d55-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-c23332-z945b\" (UID: \"ed0d0755-723c-446f-86fc-792619847d55\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c23332-z945b"
Sep 09 16:29:10.599743 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:10.599706    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c23332-z945b"
Sep 09 16:29:10.869162 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:10.869130    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c23332-z945b"]
Sep 09 16:29:10.871563 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:29:10.871535    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded0d0755_723c_446f_86fc_792619847d55.slice/crio-4909b2ec0e17008fd94b2e6524557afab83f2e7285560adf2ee271568c989337 WatchSource:0}: Error finding container 4909b2ec0e17008fd94b2e6524557afab83f2e7285560adf2ee271568c989337: Status 404 returned error can't find the container with id 4909b2ec0e17008fd94b2e6524557afab83f2e7285560adf2ee271568c989337
Sep 09 16:29:11.358095 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:11.358051    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c23332-z945b" event={"ID":"ed0d0755-723c-446f-86fc-792619847d55","Type":"ContainerStarted","Data":"2a9de9e1b66af8ea70047b16a9d4a8c2d425c9b0adc0297bdb28db62252349ce"}
Sep 09 16:29:11.358507 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:11.358120    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c23332-z945b" event={"ID":"ed0d0755-723c-446f-86fc-792619847d55","Type":"ContainerStarted","Data":"4909b2ec0e17008fd94b2e6524557afab83f2e7285560adf2ee271568c989337"}
Sep 09 16:29:11.374768 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:11.374717    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c23332-z945b" podStartSLOduration=1.3747007949999999 podStartE2EDuration="1.374700795s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 16:29:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 16:29:11.372922206 +0000 UTC m=+1226.790772228" watchObservedRunningTime="2026-09-09 16:29:11.374700795 +0000 UTC m=+1226.792550818"
Sep 09 16:29:12.364931 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:12.364894    2567 generic.go:356] "Generic (PLEG): container finished" podID="ed0d0755-723c-446f-86fc-792619847d55" containerID="2a9de9e1b66af8ea70047b16a9d4a8c2d425c9b0adc0297bdb28db62252349ce" exitCode=0
Sep 09 16:29:12.365408 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:12.364975    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c23332-z945b" event={"ID":"ed0d0755-723c-446f-86fc-792619847d55","Type":"ContainerDied","Data":"2a9de9e1b66af8ea70047b16a9d4a8c2d425c9b0adc0297bdb28db62252349ce"}
Sep 09 16:29:13.511203 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:13.511034    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c23332-z945b"
Sep 09 16:29:13.576727 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:13.576693    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed0d0755-723c-446f-86fc-792619847d55-maas-api-service-ca\") pod \"ed0d0755-723c-446f-86fc-792619847d55\" (UID: \"ed0d0755-723c-446f-86fc-792619847d55\") "
Sep 09 16:29:13.577068 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:13.577045    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed0d0755-723c-446f-86fc-792619847d55-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ed0d0755-723c-446f-86fc-792619847d55" (UID: "ed0d0755-723c-446f-86fc-792619847d55"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 16:29:13.677899 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:13.677815    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed0d0755-723c-446f-86fc-792619847d55-maas-api-service-ca\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:29:14.377537 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:14.377501    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c23332-z945b"
Sep 09 16:29:14.377714 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:14.377513    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c23332-z945b" event={"ID":"ed0d0755-723c-446f-86fc-792619847d55","Type":"ContainerDied","Data":"4909b2ec0e17008fd94b2e6524557afab83f2e7285560adf2ee271568c989337"}
Sep 09 16:29:14.377714 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:14.377621    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4909b2ec0e17008fd94b2e6524557afab83f2e7285560adf2ee271568c989337"
Sep 09 16:29:15.182630 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:15.182570    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl"
Sep 09 16:29:15.189619 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:15.189598    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bq578\" (UniqueName: \"kubernetes.io/projected/f1e36da0-3a7c-4765-aeb6-0b6339a09cc4-kube-api-access-bq578\") pod \"f1e36da0-3a7c-4765-aeb6-0b6339a09cc4\" (UID: \"f1e36da0-3a7c-4765-aeb6-0b6339a09cc4\") "
Sep 09 16:29:15.189719 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:15.189653    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1e36da0-3a7c-4765-aeb6-0b6339a09cc4-istio-podinfo\") pod \"f1e36da0-3a7c-4765-aeb6-0b6339a09cc4\" (UID: \"f1e36da0-3a7c-4765-aeb6-0b6339a09cc4\") "
Sep 09 16:29:15.189719 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:15.189679    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1e36da0-3a7c-4765-aeb6-0b6339a09cc4-workload-certs\") pod \"f1e36da0-3a7c-4765-aeb6-0b6339a09cc4\" (UID: \"f1e36da0-3a7c-4765-aeb6-0b6339a09cc4\") "
Sep 09 16:29:15.189719 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:15.189702    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1e36da0-3a7c-4765-aeb6-0b6339a09cc4-credential-socket\") pod \"f1e36da0-3a7c-4765-aeb6-0b6339a09cc4\" (UID: \"f1e36da0-3a7c-4765-aeb6-0b6339a09cc4\") "
Sep 09 16:29:15.189719 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:15.189718    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1e36da0-3a7c-4765-aeb6-0b6339a09cc4-istio-token\") pod \"f1e36da0-3a7c-4765-aeb6-0b6339a09cc4\" (UID: \"f1e36da0-3a7c-4765-aeb6-0b6339a09cc4\") "
Sep 09 16:29:15.189931 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:15.189746    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1e36da0-3a7c-4765-aeb6-0b6339a09cc4-istio-envoy\") pod \"f1e36da0-3a7c-4765-aeb6-0b6339a09cc4\" (UID: \"f1e36da0-3a7c-4765-aeb6-0b6339a09cc4\") "
Sep 09 16:29:15.189931 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:15.189769    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1e36da0-3a7c-4765-aeb6-0b6339a09cc4-workload-socket\") pod \"f1e36da0-3a7c-4765-aeb6-0b6339a09cc4\" (UID: \"f1e36da0-3a7c-4765-aeb6-0b6339a09cc4\") "
Sep 09 16:29:15.189931 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:15.189784    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1e36da0-3a7c-4765-aeb6-0b6339a09cc4-istio-data\") pod \"f1e36da0-3a7c-4765-aeb6-0b6339a09cc4\" (UID: \"f1e36da0-3a7c-4765-aeb6-0b6339a09cc4\") "
Sep 09 16:29:15.189931 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:15.189803    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1e36da0-3a7c-4765-aeb6-0b6339a09cc4-istiod-ca-cert\") pod \"f1e36da0-3a7c-4765-aeb6-0b6339a09cc4\" (UID: \"f1e36da0-3a7c-4765-aeb6-0b6339a09cc4\") "
Sep 09 16:29:15.190186 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:15.190009    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1e36da0-3a7c-4765-aeb6-0b6339a09cc4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f1e36da0-3a7c-4765-aeb6-0b6339a09cc4" (UID: "f1e36da0-3a7c-4765-aeb6-0b6339a09cc4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:29:15.190186 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:15.190128    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1e36da0-3a7c-4765-aeb6-0b6339a09cc4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f1e36da0-3a7c-4765-aeb6-0b6339a09cc4" (UID: "f1e36da0-3a7c-4765-aeb6-0b6339a09cc4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:29:15.190186 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:15.190125    2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1e36da0-3a7c-4765-aeb6-0b6339a09cc4-workload-certs\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:29:15.190364 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:15.190344    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1e36da0-3a7c-4765-aeb6-0b6339a09cc4-istio-data" (OuterVolumeSpecName: "istio-data") pod "f1e36da0-3a7c-4765-aeb6-0b6339a09cc4" (UID: "f1e36da0-3a7c-4765-aeb6-0b6339a09cc4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:29:15.190420 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:15.190363    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1e36da0-3a7c-4765-aeb6-0b6339a09cc4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f1e36da0-3a7c-4765-aeb6-0b6339a09cc4" (UID: "f1e36da0-3a7c-4765-aeb6-0b6339a09cc4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 16:29:15.190546 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:15.190484    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1e36da0-3a7c-4765-aeb6-0b6339a09cc4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f1e36da0-3a7c-4765-aeb6-0b6339a09cc4" (UID: "f1e36da0-3a7c-4765-aeb6-0b6339a09cc4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:29:15.192030 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:15.192006    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1e36da0-3a7c-4765-aeb6-0b6339a09cc4-kube-api-access-bq578" (OuterVolumeSpecName: "kube-api-access-bq578") pod "f1e36da0-3a7c-4765-aeb6-0b6339a09cc4" (UID: "f1e36da0-3a7c-4765-aeb6-0b6339a09cc4"). InnerVolumeSpecName "kube-api-access-bq578". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 16:29:15.192256 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:15.192224    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1e36da0-3a7c-4765-aeb6-0b6339a09cc4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f1e36da0-3a7c-4765-aeb6-0b6339a09cc4" (UID: "f1e36da0-3a7c-4765-aeb6-0b6339a09cc4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:29:15.192503 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:15.192486    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1e36da0-3a7c-4765-aeb6-0b6339a09cc4-istio-token" (OuterVolumeSpecName: "istio-token") pod "f1e36da0-3a7c-4765-aeb6-0b6339a09cc4" (UID: "f1e36da0-3a7c-4765-aeb6-0b6339a09cc4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 16:29:15.192555 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:15.192510    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f1e36da0-3a7c-4765-aeb6-0b6339a09cc4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f1e36da0-3a7c-4765-aeb6-0b6339a09cc4" (UID: "f1e36da0-3a7c-4765-aeb6-0b6339a09cc4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 16:29:15.290595 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:15.290560    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bq578\" (UniqueName: \"kubernetes.io/projected/f1e36da0-3a7c-4765-aeb6-0b6339a09cc4-kube-api-access-bq578\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:29:15.290595 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:15.290589    2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1e36da0-3a7c-4765-aeb6-0b6339a09cc4-istio-podinfo\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:29:15.290595 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:15.290599    2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1e36da0-3a7c-4765-aeb6-0b6339a09cc4-credential-socket\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:29:15.290838 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:15.290610    2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1e36da0-3a7c-4765-aeb6-0b6339a09cc4-istio-token\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:29:15.290838 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:15.290618    2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1e36da0-3a7c-4765-aeb6-0b6339a09cc4-istio-envoy\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:29:15.290838 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:15.290625    2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1e36da0-3a7c-4765-aeb6-0b6339a09cc4-workload-socket\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:29:15.290838 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:15.290633    2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1e36da0-3a7c-4765-aeb6-0b6339a09cc4-istio-data\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:29:15.290838 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:15.290640    2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1e36da0-3a7c-4765-aeb6-0b6339a09cc4-istiod-ca-cert\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:29:15.383849 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:15.383814    2567 generic.go:356] "Generic (PLEG): container finished" podID="f1e36da0-3a7c-4765-aeb6-0b6339a09cc4" containerID="75a99bdfb20bee38d1ed434e26628356d776f38035872feefd7b50f587a07fe1" exitCode=0
Sep 09 16:29:15.384024 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:15.383870    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl"
Sep 09 16:29:15.384024 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:15.383896    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl" event={"ID":"f1e36da0-3a7c-4765-aeb6-0b6339a09cc4","Type":"ContainerDied","Data":"75a99bdfb20bee38d1ed434e26628356d776f38035872feefd7b50f587a07fe1"}
Sep 09 16:29:15.384024 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:15.383954    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl" event={"ID":"f1e36da0-3a7c-4765-aeb6-0b6339a09cc4","Type":"ContainerDied","Data":"9b28b2b656187d62207b5afa3d39cb901eceb336cb9e2ca49b526fb944a02c06"}
Sep 09 16:29:15.384024 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:15.383979    2567 scope.go:117] "RemoveContainer" containerID="75a99bdfb20bee38d1ed434e26628356d776f38035872feefd7b50f587a07fe1"
Sep 09 16:29:15.393356 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:15.393337    2567 scope.go:117] "RemoveContainer" containerID="6ddf2705b7bd4446dc2110af85a6252fde2d8f77b3c9729524b6a6a6c72f880b"
Sep 09 16:29:15.401586 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:15.401565    2567 scope.go:117] "RemoveContainer" containerID="75a99bdfb20bee38d1ed434e26628356d776f38035872feefd7b50f587a07fe1"
Sep 09 16:29:15.401899 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:29:15.401880    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75a99bdfb20bee38d1ed434e26628356d776f38035872feefd7b50f587a07fe1\": container with ID starting with 75a99bdfb20bee38d1ed434e26628356d776f38035872feefd7b50f587a07fe1 not found: ID does not exist" containerID="75a99bdfb20bee38d1ed434e26628356d776f38035872feefd7b50f587a07fe1"
Sep 09 16:29:15.401954 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:15.401907    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75a99bdfb20bee38d1ed434e26628356d776f38035872feefd7b50f587a07fe1"} err="failed to get container status \"75a99bdfb20bee38d1ed434e26628356d776f38035872feefd7b50f587a07fe1\": rpc error: code = NotFound desc = could not find container \"75a99bdfb20bee38d1ed434e26628356d776f38035872feefd7b50f587a07fe1\": container with ID starting with 75a99bdfb20bee38d1ed434e26628356d776f38035872feefd7b50f587a07fe1 not found: ID does not exist"
Sep 09 16:29:15.401954 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:15.401923    2567 scope.go:117] "RemoveContainer" containerID="6ddf2705b7bd4446dc2110af85a6252fde2d8f77b3c9729524b6a6a6c72f880b"
Sep 09 16:29:15.402173 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:29:15.402153    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ddf2705b7bd4446dc2110af85a6252fde2d8f77b3c9729524b6a6a6c72f880b\": container with ID starting with 6ddf2705b7bd4446dc2110af85a6252fde2d8f77b3c9729524b6a6a6c72f880b not found: ID does not exist" containerID="6ddf2705b7bd4446dc2110af85a6252fde2d8f77b3c9729524b6a6a6c72f880b"
Sep 09 16:29:15.402233 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:15.402183    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ddf2705b7bd4446dc2110af85a6252fde2d8f77b3c9729524b6a6a6c72f880b"} err="failed to get container status \"6ddf2705b7bd4446dc2110af85a6252fde2d8f77b3c9729524b6a6a6c72f880b\": rpc error: code = NotFound desc = could not find container \"6ddf2705b7bd4446dc2110af85a6252fde2d8f77b3c9729524b6a6a6c72f880b\": container with ID starting with 6ddf2705b7bd4446dc2110af85a6252fde2d8f77b3c9729524b6a6a6c72f880b not found: ID does not exist"
Sep 09 16:29:15.406505 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:15.406431    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl"]
Sep 09 16:29:15.412684 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:15.412659    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-6d44b6-openshift-default-7f9b4bdbdb-zc2nl"]
Sep 09 16:29:17.035791 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:17.035760    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1e36da0-3a7c-4765-aeb6-0b6339a09cc4" path="/var/lib/kubelet/pods/f1e36da0-3a7c-4765-aeb6-0b6339a09cc4/volumes"
Sep 09 16:29:17.741545 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:17.741513    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-13b02364-c4c4b69d8-b8hnv"]
Sep 09 16:29:17.741937 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:17.741924    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1e36da0-3a7c-4765-aeb6-0b6339a09cc4" containerName="istio-proxy"
Sep 09 16:29:17.741992 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:17.741940    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1e36da0-3a7c-4765-aeb6-0b6339a09cc4" containerName="istio-proxy"
Sep 09 16:29:17.741992 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:17.741950    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed0d0755-723c-446f-86fc-792619847d55" containerName="revoke-keys"
Sep 09 16:29:17.741992 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:17.741957    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed0d0755-723c-446f-86fc-792619847d55" containerName="revoke-keys"
Sep 09 16:29:17.741992 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:17.741971    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1e36da0-3a7c-4765-aeb6-0b6339a09cc4" containerName="istio-proxy"
Sep 09 16:29:17.741992 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:17.741976    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1e36da0-3a7c-4765-aeb6-0b6339a09cc4" containerName="istio-proxy"
Sep 09 16:29:17.742154 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:17.742026    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed0d0755-723c-446f-86fc-792619847d55" containerName="revoke-keys"
Sep 09 16:29:17.742154 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:17.742036    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1e36da0-3a7c-4765-aeb6-0b6339a09cc4" containerName="istio-proxy"
Sep 09 16:29:17.742154 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:17.742047    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1e36da0-3a7c-4765-aeb6-0b6339a09cc4" containerName="istio-proxy"
Sep 09 16:29:17.746674 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:17.746655    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-13b02364-c4c4b69d8-b8hnv"
Sep 09 16:29:17.748966 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:17.748943    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-13b02364-serving-cert\""
Sep 09 16:29:17.749106 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:17.749011    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 09 16:29:17.750392 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:17.750361    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 09 16:29:17.750551 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:17.750370    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-mslnv\""
Sep 09 16:29:17.750551 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:17.750371    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 09 16:29:17.756335 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:17.756295    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-13b02364-c4c4b69d8-b8hnv"]
Sep 09 16:29:17.810904 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:17.810854    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6aa3d61-5754-45bd-a397-0f997d1310bf-maas-api-tls\") pod \"maas-api-e2e-mt-13b02364-c4c4b69d8-b8hnv\" (UID: \"d6aa3d61-5754-45bd-a397-0f997d1310bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-13b02364-c4c4b69d8-b8hnv"
Sep 09 16:29:17.811100 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:17.810932    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6aa3d61-5754-45bd-a397-0f997d1310bf-metrics-certs\") pod \"maas-api-e2e-mt-13b02364-c4c4b69d8-b8hnv\" (UID: \"d6aa3d61-5754-45bd-a397-0f997d1310bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-13b02364-c4c4b69d8-b8hnv"
Sep 09 16:29:17.811100 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:17.811006    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxd7s\" (UniqueName: \"kubernetes.io/projected/d6aa3d61-5754-45bd-a397-0f997d1310bf-kube-api-access-vxd7s\") pod \"maas-api-e2e-mt-13b02364-c4c4b69d8-b8hnv\" (UID: \"d6aa3d61-5754-45bd-a397-0f997d1310bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-13b02364-c4c4b69d8-b8hnv"
Sep 09 16:29:17.811100 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:17.811036    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6aa3d61-5754-45bd-a397-0f997d1310bf-openshift-service-ca\") pod \"maas-api-e2e-mt-13b02364-c4c4b69d8-b8hnv\" (UID: \"d6aa3d61-5754-45bd-a397-0f997d1310bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-13b02364-c4c4b69d8-b8hnv"
Sep 09 16:29:17.911488 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:17.911418    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6aa3d61-5754-45bd-a397-0f997d1310bf-maas-api-tls\") pod \"maas-api-e2e-mt-13b02364-c4c4b69d8-b8hnv\" (UID: \"d6aa3d61-5754-45bd-a397-0f997d1310bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-13b02364-c4c4b69d8-b8hnv"
Sep 09 16:29:17.911702 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:17.911528    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6aa3d61-5754-45bd-a397-0f997d1310bf-metrics-certs\") pod \"maas-api-e2e-mt-13b02364-c4c4b69d8-b8hnv\" (UID: \"d6aa3d61-5754-45bd-a397-0f997d1310bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-13b02364-c4c4b69d8-b8hnv"
Sep 09 16:29:17.911702 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:29:17.911600    2567 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-13b02364-serving-cert: secret "maas-api-e2e-mt-13b02364-serving-cert" not found
Sep 09 16:29:17.911702 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:17.911606    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxd7s\" (UniqueName: \"kubernetes.io/projected/d6aa3d61-5754-45bd-a397-0f997d1310bf-kube-api-access-vxd7s\") pod \"maas-api-e2e-mt-13b02364-c4c4b69d8-b8hnv\" (UID: \"d6aa3d61-5754-45bd-a397-0f997d1310bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-13b02364-c4c4b69d8-b8hnv"
Sep 09 16:29:17.911702 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:29:17.911687    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6aa3d61-5754-45bd-a397-0f997d1310bf-maas-api-tls podName:d6aa3d61-5754-45bd-a397-0f997d1310bf nodeName:}" failed. No retries permitted until 2026-09-09 16:29:18.411662263 +0000 UTC m=+1233.829512240 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d6aa3d61-5754-45bd-a397-0f997d1310bf-maas-api-tls") pod "maas-api-e2e-mt-13b02364-c4c4b69d8-b8hnv" (UID: "d6aa3d61-5754-45bd-a397-0f997d1310bf") : secret "maas-api-e2e-mt-13b02364-serving-cert" not found
Sep 09 16:29:17.911930 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:17.911737    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6aa3d61-5754-45bd-a397-0f997d1310bf-openshift-service-ca\") pod \"maas-api-e2e-mt-13b02364-c4c4b69d8-b8hnv\" (UID: \"d6aa3d61-5754-45bd-a397-0f997d1310bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-13b02364-c4c4b69d8-b8hnv"
Sep 09 16:29:17.912325 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:17.912306    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6aa3d61-5754-45bd-a397-0f997d1310bf-openshift-service-ca\") pod \"maas-api-e2e-mt-13b02364-c4c4b69d8-b8hnv\" (UID: \"d6aa3d61-5754-45bd-a397-0f997d1310bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-13b02364-c4c4b69d8-b8hnv"
Sep 09 16:29:17.914232 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:17.914215    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6aa3d61-5754-45bd-a397-0f997d1310bf-metrics-certs\") pod \"maas-api-e2e-mt-13b02364-c4c4b69d8-b8hnv\" (UID: \"d6aa3d61-5754-45bd-a397-0f997d1310bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-13b02364-c4c4b69d8-b8hnv"
Sep 09 16:29:17.920000 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:17.919981    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxd7s\" (UniqueName: \"kubernetes.io/projected/d6aa3d61-5754-45bd-a397-0f997d1310bf-kube-api-access-vxd7s\") pod \"maas-api-e2e-mt-13b02364-c4c4b69d8-b8hnv\" (UID: \"d6aa3d61-5754-45bd-a397-0f997d1310bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-13b02364-c4c4b69d8-b8hnv"
Sep 09 16:29:18.417409 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:18.417373    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6aa3d61-5754-45bd-a397-0f997d1310bf-maas-api-tls\") pod \"maas-api-e2e-mt-13b02364-c4c4b69d8-b8hnv\" (UID: \"d6aa3d61-5754-45bd-a397-0f997d1310bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-13b02364-c4c4b69d8-b8hnv"
Sep 09 16:29:18.420268 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:18.420241    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6aa3d61-5754-45bd-a397-0f997d1310bf-maas-api-tls\") pod \"maas-api-e2e-mt-13b02364-c4c4b69d8-b8hnv\" (UID: \"d6aa3d61-5754-45bd-a397-0f997d1310bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-13b02364-c4c4b69d8-b8hnv"
Sep 09 16:29:18.661619 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:18.661579    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-13b02364-c4c4b69d8-b8hnv"
Sep 09 16:29:18.932291 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:18.932223    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-13b02364-c4c4b69d8-b8hnv"]
Sep 09 16:29:18.934422 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:29:18.934393    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6aa3d61_5754_45bd_a397_0f997d1310bf.slice/crio-adbed8b5d1983d92a626cabb5e702be6e43ae983eebf1dad94ce01c8da63737f WatchSource:0}: Error finding container adbed8b5d1983d92a626cabb5e702be6e43ae983eebf1dad94ce01c8da63737f: Status 404 returned error can't find the container with id adbed8b5d1983d92a626cabb5e702be6e43ae983eebf1dad94ce01c8da63737f
Sep 09 16:29:19.408144 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:19.408106    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-13b02364-c4c4b69d8-b8hnv" event={"ID":"d6aa3d61-5754-45bd-a397-0f997d1310bf","Type":"ContainerStarted","Data":"adbed8b5d1983d92a626cabb5e702be6e43ae983eebf1dad94ce01c8da63737f"}
Sep 09 16:29:21.216717 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:21.216675    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:29:21.295932 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:21.295903    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c23332-z945b"]
Sep 09 16:29:21.298971 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:21.298948    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c23332-z945b"]
Sep 09 16:29:21.423573 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:21.423439    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-13b02364-c4c4b69d8-b8hnv" event={"ID":"d6aa3d61-5754-45bd-a397-0f997d1310bf","Type":"ContainerStarted","Data":"bc2abbebed5b42173171a5b718eac0aa1bb28ab8b04a2378db26172cff8b21a0"}
Sep 09 16:29:21.423573 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:21.423537    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-13b02364-c4c4b69d8-b8hnv"
Sep 09 16:29:21.447981 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:21.447926    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-13b02364-c4c4b69d8-b8hnv" podStartSLOduration=2.107169475 podStartE2EDuration="4.44790607s" totalImagesPullingTime="2.340736595s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 16:29:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 16:29:21.445732178 +0000 UTC m=+1236.863582178" watchObservedRunningTime="2026-09-09 16:29:21.44790607 +0000 UTC m=+1236.865756071"
Sep 09 16:29:23.035832 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:23.035791    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed0d0755-723c-446f-86fc-792619847d55" path="/var/lib/kubelet/pods/ed0d0755-723c-446f-86fc-792619847d55/volumes"
Sep 09 16:29:27.441018 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:27.440984    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-13b02364-c4c4b69d8-b8hnv"
Sep 09 16:29:27.671640 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:27.671596    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc"]
Sep 09 16:29:27.671911 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:27.671858    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc" podUID="155986b3-9283-4eea-bff1-479cad0f546f" containerName="istio-proxy" containerID="cri-o://908d3524f14cfafda38ea73492182cda3b7bf663ce8ba126225eb3b43daac779" gracePeriod=30
Sep 09 16:29:32.935231 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:32.935160    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc"
Sep 09 16:29:33.056708 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:33.056672    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/155986b3-9283-4eea-bff1-479cad0f546f-workload-socket\") pod \"155986b3-9283-4eea-bff1-479cad0f546f\" (UID: \"155986b3-9283-4eea-bff1-479cad0f546f\") "
Sep 09 16:29:33.056872 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:33.056729    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/155986b3-9283-4eea-bff1-479cad0f546f-istiod-ca-cert\") pod \"155986b3-9283-4eea-bff1-479cad0f546f\" (UID: \"155986b3-9283-4eea-bff1-479cad0f546f\") "
Sep 09 16:29:33.056872 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:33.056784    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5wcs9\" (UniqueName: \"kubernetes.io/projected/155986b3-9283-4eea-bff1-479cad0f546f-kube-api-access-5wcs9\") pod \"155986b3-9283-4eea-bff1-479cad0f546f\" (UID: \"155986b3-9283-4eea-bff1-479cad0f546f\") "
Sep 09 16:29:33.056872 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:33.056831    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/155986b3-9283-4eea-bff1-479cad0f546f-istio-token\") pod \"155986b3-9283-4eea-bff1-479cad0f546f\" (UID: \"155986b3-9283-4eea-bff1-479cad0f546f\") "
Sep 09 16:29:33.057010 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:33.056894    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/155986b3-9283-4eea-bff1-479cad0f546f-istio-data\") pod \"155986b3-9283-4eea-bff1-479cad0f546f\" (UID: \"155986b3-9283-4eea-bff1-479cad0f546f\") "
Sep 09 16:29:33.057010 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:33.056925    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/155986b3-9283-4eea-bff1-479cad0f546f-istio-podinfo\") pod \"155986b3-9283-4eea-bff1-479cad0f546f\" (UID: \"155986b3-9283-4eea-bff1-479cad0f546f\") "
Sep 09 16:29:33.057010 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:33.056972    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/155986b3-9283-4eea-bff1-479cad0f546f-workload-certs\") pod \"155986b3-9283-4eea-bff1-479cad0f546f\" (UID: \"155986b3-9283-4eea-bff1-479cad0f546f\") "
Sep 09 16:29:33.057010 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:33.056991    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/155986b3-9283-4eea-bff1-479cad0f546f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "155986b3-9283-4eea-bff1-479cad0f546f" (UID: "155986b3-9283-4eea-bff1-479cad0f546f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:29:33.057010 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:33.057006    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/155986b3-9283-4eea-bff1-479cad0f546f-istio-envoy\") pod \"155986b3-9283-4eea-bff1-479cad0f546f\" (UID: \"155986b3-9283-4eea-bff1-479cad0f546f\") "
Sep 09 16:29:33.057259 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:33.057031    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/155986b3-9283-4eea-bff1-479cad0f546f-credential-socket\") pod \"155986b3-9283-4eea-bff1-479cad0f546f\" (UID: \"155986b3-9283-4eea-bff1-479cad0f546f\") "
Sep 09 16:29:33.057259 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:33.057228    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/155986b3-9283-4eea-bff1-479cad0f546f-istio-data" (OuterVolumeSpecName: "istio-data") pod "155986b3-9283-4eea-bff1-479cad0f546f" (UID: "155986b3-9283-4eea-bff1-479cad0f546f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:29:33.057259 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:33.057137    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/155986b3-9283-4eea-bff1-479cad0f546f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "155986b3-9283-4eea-bff1-479cad0f546f" (UID: "155986b3-9283-4eea-bff1-479cad0f546f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 16:29:33.057259 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:33.057240    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/155986b3-9283-4eea-bff1-479cad0f546f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "155986b3-9283-4eea-bff1-479cad0f546f" (UID: "155986b3-9283-4eea-bff1-479cad0f546f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:29:33.057487 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:33.057352    2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/155986b3-9283-4eea-bff1-479cad0f546f-workload-certs\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:29:33.057487 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:33.057398    2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/155986b3-9283-4eea-bff1-479cad0f546f-workload-socket\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:29:33.057487 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:33.057412    2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/155986b3-9283-4eea-bff1-479cad0f546f-istiod-ca-cert\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:29:33.057487 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:33.057426    2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/155986b3-9283-4eea-bff1-479cad0f546f-istio-data\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:29:33.057487 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:33.057431    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/155986b3-9283-4eea-bff1-479cad0f546f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "155986b3-9283-4eea-bff1-479cad0f546f" (UID: "155986b3-9283-4eea-bff1-479cad0f546f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:29:33.059254 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:33.059229    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/155986b3-9283-4eea-bff1-479cad0f546f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "155986b3-9283-4eea-bff1-479cad0f546f" (UID: "155986b3-9283-4eea-bff1-479cad0f546f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:29:33.059337 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:33.059276    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/155986b3-9283-4eea-bff1-479cad0f546f-istio-token" (OuterVolumeSpecName: "istio-token") pod "155986b3-9283-4eea-bff1-479cad0f546f" (UID: "155986b3-9283-4eea-bff1-479cad0f546f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 16:29:33.059719 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:33.059697    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/155986b3-9283-4eea-bff1-479cad0f546f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "155986b3-9283-4eea-bff1-479cad0f546f" (UID: "155986b3-9283-4eea-bff1-479cad0f546f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 16:29:33.059805 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:33.059723    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/155986b3-9283-4eea-bff1-479cad0f546f-kube-api-access-5wcs9" (OuterVolumeSpecName: "kube-api-access-5wcs9") pod "155986b3-9283-4eea-bff1-479cad0f546f" (UID: "155986b3-9283-4eea-bff1-479cad0f546f"). InnerVolumeSpecName "kube-api-access-5wcs9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 16:29:33.158696 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:33.158660    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5wcs9\" (UniqueName: \"kubernetes.io/projected/155986b3-9283-4eea-bff1-479cad0f546f-kube-api-access-5wcs9\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:29:33.158696 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:33.158690    2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/155986b3-9283-4eea-bff1-479cad0f546f-istio-token\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:29:33.158696 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:33.158700    2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/155986b3-9283-4eea-bff1-479cad0f546f-istio-podinfo\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:29:33.158696 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:33.158708    2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/155986b3-9283-4eea-bff1-479cad0f546f-istio-envoy\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:29:33.158971 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:33.158717    2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/155986b3-9283-4eea-bff1-479cad0f546f-credential-socket\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:29:33.501284 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:33.501244    2567 generic.go:356] "Generic (PLEG): container finished" podID="155986b3-9283-4eea-bff1-479cad0f546f" containerID="908d3524f14cfafda38ea73492182cda3b7bf663ce8ba126225eb3b43daac779" exitCode=0
Sep 09 16:29:33.501473 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:33.501309    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc"
Sep 09 16:29:33.501473 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:33.501321    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc" event={"ID":"155986b3-9283-4eea-bff1-479cad0f546f","Type":"ContainerDied","Data":"908d3524f14cfafda38ea73492182cda3b7bf663ce8ba126225eb3b43daac779"}
Sep 09 16:29:33.501473 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:33.501417    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc" event={"ID":"155986b3-9283-4eea-bff1-479cad0f546f","Type":"ContainerDied","Data":"e6d5585416941cfe24360e4a92eb868f53a48d3a164b0e6e8fdfa11a3fdc521d"}
Sep 09 16:29:33.501473 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:33.501446    2567 scope.go:117] "RemoveContainer" containerID="908d3524f14cfafda38ea73492182cda3b7bf663ce8ba126225eb3b43daac779"
Sep 09 16:29:33.511267 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:33.511249    2567 scope.go:117] "RemoveContainer" containerID="96fc12418897c31e2d4ce3ebed36ff24c906ae77699a6aed090a6dc1676f41b2"
Sep 09 16:29:33.519783 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:33.519763    2567 scope.go:117] "RemoveContainer" containerID="908d3524f14cfafda38ea73492182cda3b7bf663ce8ba126225eb3b43daac779"
Sep 09 16:29:33.520078 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:29:33.520058    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"908d3524f14cfafda38ea73492182cda3b7bf663ce8ba126225eb3b43daac779\": container with ID starting with 908d3524f14cfafda38ea73492182cda3b7bf663ce8ba126225eb3b43daac779 not found: ID does not exist" containerID="908d3524f14cfafda38ea73492182cda3b7bf663ce8ba126225eb3b43daac779"
Sep 09 16:29:33.520153 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:33.520091    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"908d3524f14cfafda38ea73492182cda3b7bf663ce8ba126225eb3b43daac779"} err="failed to get container status \"908d3524f14cfafda38ea73492182cda3b7bf663ce8ba126225eb3b43daac779\": rpc error: code = NotFound desc = could not find container \"908d3524f14cfafda38ea73492182cda3b7bf663ce8ba126225eb3b43daac779\": container with ID starting with 908d3524f14cfafda38ea73492182cda3b7bf663ce8ba126225eb3b43daac779 not found: ID does not exist"
Sep 09 16:29:33.520153 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:33.520117    2567 scope.go:117] "RemoveContainer" containerID="96fc12418897c31e2d4ce3ebed36ff24c906ae77699a6aed090a6dc1676f41b2"
Sep 09 16:29:33.520387 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:29:33.520370    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96fc12418897c31e2d4ce3ebed36ff24c906ae77699a6aed090a6dc1676f41b2\": container with ID starting with 96fc12418897c31e2d4ce3ebed36ff24c906ae77699a6aed090a6dc1676f41b2 not found: ID does not exist" containerID="96fc12418897c31e2d4ce3ebed36ff24c906ae77699a6aed090a6dc1676f41b2"
Sep 09 16:29:33.520423 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:33.520393    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96fc12418897c31e2d4ce3ebed36ff24c906ae77699a6aed090a6dc1676f41b2"} err="failed to get container status \"96fc12418897c31e2d4ce3ebed36ff24c906ae77699a6aed090a6dc1676f41b2\": rpc error: code = NotFound desc = could not find container \"96fc12418897c31e2d4ce3ebed36ff24c906ae77699a6aed090a6dc1676f41b2\": container with ID starting with 96fc12418897c31e2d4ce3ebed36ff24c906ae77699a6aed090a6dc1676f41b2 not found: ID does not exist"
Sep 09 16:29:33.524245 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:33.524223    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc"]
Sep 09 16:29:33.531129 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:33.531106    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-c23332-openshift-default-7f67665888-ccvsc"]
Sep 09 16:29:35.036094 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:35.036060    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="155986b3-9283-4eea-bff1-479cad0f546f" path="/var/lib/kubelet/pods/155986b3-9283-4eea-bff1-479cad0f546f/volumes"
Sep 09 16:29:37.826178 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:37.826104    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:29:57.320603 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:57.320568    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl"]
Sep 09 16:29:57.321000 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:57.320942    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="155986b3-9283-4eea-bff1-479cad0f546f" containerName="istio-proxy"
Sep 09 16:29:57.321000 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:57.320953    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="155986b3-9283-4eea-bff1-479cad0f546f" containerName="istio-proxy"
Sep 09 16:29:57.321000 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:57.320966    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="155986b3-9283-4eea-bff1-479cad0f546f" containerName="istio-proxy"
Sep 09 16:29:57.321000 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:57.320973    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="155986b3-9283-4eea-bff1-479cad0f546f" containerName="istio-proxy"
Sep 09 16:29:57.321147 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:57.321043    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="155986b3-9283-4eea-bff1-479cad0f546f" containerName="istio-proxy"
Sep 09 16:29:57.321147 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:57.321054    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="155986b3-9283-4eea-bff1-479cad0f546f" containerName="istio-proxy"
Sep 09 16:29:57.327220 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:57.327198    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl"
Sep 09 16:29:57.329388 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:57.329364    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-9520a4-openshift-default-dockercfg-svv9g\""
Sep 09 16:29:57.337138 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:57.337114    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl"]
Sep 09 16:29:57.376991 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:57.376946    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/993e8512-0a13-4565-8fd6-53d9f4910d95-credential-socket\") pod \"e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl\" (UID: \"993e8512-0a13-4565-8fd6-53d9f4910d95\") " pod="openshift-ingress/e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl"
Sep 09 16:29:57.377154 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:57.377015    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/993e8512-0a13-4565-8fd6-53d9f4910d95-istio-data\") pod \"e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl\" (UID: \"993e8512-0a13-4565-8fd6-53d9f4910d95\") " pod="openshift-ingress/e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl"
Sep 09 16:29:57.377154 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:57.377056    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/993e8512-0a13-4565-8fd6-53d9f4910d95-workload-certs\") pod \"e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl\" (UID: \"993e8512-0a13-4565-8fd6-53d9f4910d95\") " pod="openshift-ingress/e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl"
Sep 09 16:29:57.377154 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:57.377115    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/993e8512-0a13-4565-8fd6-53d9f4910d95-istio-podinfo\") pod \"e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl\" (UID: \"993e8512-0a13-4565-8fd6-53d9f4910d95\") " pod="openshift-ingress/e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl"
Sep 09 16:29:57.377154 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:57.377143    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rk29\" (UniqueName: \"kubernetes.io/projected/993e8512-0a13-4565-8fd6-53d9f4910d95-kube-api-access-9rk29\") pod \"e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl\" (UID: \"993e8512-0a13-4565-8fd6-53d9f4910d95\") " pod="openshift-ingress/e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl"
Sep 09 16:29:57.377367 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:57.377278    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/993e8512-0a13-4565-8fd6-53d9f4910d95-istio-token\") pod \"e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl\" (UID: \"993e8512-0a13-4565-8fd6-53d9f4910d95\") " pod="openshift-ingress/e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl"
Sep 09 16:29:57.377367 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:57.377312    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/993e8512-0a13-4565-8fd6-53d9f4910d95-istiod-ca-cert\") pod \"e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl\" (UID: \"993e8512-0a13-4565-8fd6-53d9f4910d95\") " pod="openshift-ingress/e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl"
Sep 09 16:29:57.377367 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:57.377341    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/993e8512-0a13-4565-8fd6-53d9f4910d95-workload-socket\") pod \"e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl\" (UID: \"993e8512-0a13-4565-8fd6-53d9f4910d95\") " pod="openshift-ingress/e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl"
Sep 09 16:29:57.377533 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:57.377382    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/993e8512-0a13-4565-8fd6-53d9f4910d95-istio-envoy\") pod \"e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl\" (UID: \"993e8512-0a13-4565-8fd6-53d9f4910d95\") " pod="openshift-ingress/e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl"
Sep 09 16:29:57.477839 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:57.477798    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/993e8512-0a13-4565-8fd6-53d9f4910d95-workload-certs\") pod \"e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl\" (UID: \"993e8512-0a13-4565-8fd6-53d9f4910d95\") " pod="openshift-ingress/e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl"
Sep 09 16:29:57.478043 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:57.477850    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/993e8512-0a13-4565-8fd6-53d9f4910d95-istio-podinfo\") pod \"e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl\" (UID: \"993e8512-0a13-4565-8fd6-53d9f4910d95\") " pod="openshift-ingress/e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl"
Sep 09 16:29:57.478043 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:57.477970    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rk29\" (UniqueName: \"kubernetes.io/projected/993e8512-0a13-4565-8fd6-53d9f4910d95-kube-api-access-9rk29\") pod \"e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl\" (UID: \"993e8512-0a13-4565-8fd6-53d9f4910d95\") " pod="openshift-ingress/e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl"
Sep 09 16:29:57.478167 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:57.478060    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/993e8512-0a13-4565-8fd6-53d9f4910d95-istio-token\") pod \"e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl\" (UID: \"993e8512-0a13-4565-8fd6-53d9f4910d95\") " pod="openshift-ingress/e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl"
Sep 09 16:29:57.478167 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:57.478077    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/993e8512-0a13-4565-8fd6-53d9f4910d95-istiod-ca-cert\") pod \"e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl\" (UID: \"993e8512-0a13-4565-8fd6-53d9f4910d95\") " pod="openshift-ingress/e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl"
Sep 09 16:29:57.478167 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:57.478093    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/993e8512-0a13-4565-8fd6-53d9f4910d95-workload-socket\") pod \"e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl\" (UID: \"993e8512-0a13-4565-8fd6-53d9f4910d95\") " pod="openshift-ingress/e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl"
Sep 09 16:29:57.478167 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:57.478115    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/993e8512-0a13-4565-8fd6-53d9f4910d95-istio-envoy\") pod \"e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl\" (UID: \"993e8512-0a13-4565-8fd6-53d9f4910d95\") " pod="openshift-ingress/e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl"
Sep 09 16:29:57.478167 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:57.478157    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/993e8512-0a13-4565-8fd6-53d9f4910d95-credential-socket\") pod \"e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl\" (UID: \"993e8512-0a13-4565-8fd6-53d9f4910d95\") " pod="openshift-ingress/e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl"
Sep 09 16:29:57.478421 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:57.478170    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/993e8512-0a13-4565-8fd6-53d9f4910d95-workload-certs\") pod \"e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl\" (UID: \"993e8512-0a13-4565-8fd6-53d9f4910d95\") " pod="openshift-ingress/e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl"
Sep 09 16:29:57.478421 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:57.478320    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/993e8512-0a13-4565-8fd6-53d9f4910d95-istio-data\") pod \"e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl\" (UID: \"993e8512-0a13-4565-8fd6-53d9f4910d95\") " pod="openshift-ingress/e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl"
Sep 09 16:29:57.478568 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:57.478516    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/993e8512-0a13-4565-8fd6-53d9f4910d95-workload-socket\") pod \"e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl\" (UID: \"993e8512-0a13-4565-8fd6-53d9f4910d95\") " pod="openshift-ingress/e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl"
Sep 09 16:29:57.478631 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:57.478563    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/993e8512-0a13-4565-8fd6-53d9f4910d95-credential-socket\") pod \"e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl\" (UID: \"993e8512-0a13-4565-8fd6-53d9f4910d95\") " pod="openshift-ingress/e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl"
Sep 09 16:29:57.478755 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:57.478709    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/993e8512-0a13-4565-8fd6-53d9f4910d95-istio-data\") pod \"e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl\" (UID: \"993e8512-0a13-4565-8fd6-53d9f4910d95\") " pod="openshift-ingress/e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl"
Sep 09 16:29:57.478908 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:57.478825    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/993e8512-0a13-4565-8fd6-53d9f4910d95-istiod-ca-cert\") pod \"e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl\" (UID: \"993e8512-0a13-4565-8fd6-53d9f4910d95\") " pod="openshift-ingress/e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl"
Sep 09 16:29:57.480492 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:57.480449    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/993e8512-0a13-4565-8fd6-53d9f4910d95-istio-envoy\") pod \"e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl\" (UID: \"993e8512-0a13-4565-8fd6-53d9f4910d95\") " pod="openshift-ingress/e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl"
Sep 09 16:29:57.480657 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:57.480641    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/993e8512-0a13-4565-8fd6-53d9f4910d95-istio-podinfo\") pod \"e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl\" (UID: \"993e8512-0a13-4565-8fd6-53d9f4910d95\") " pod="openshift-ingress/e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl"
Sep 09 16:29:57.485546 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:57.485521    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rk29\" (UniqueName: \"kubernetes.io/projected/993e8512-0a13-4565-8fd6-53d9f4910d95-kube-api-access-9rk29\") pod \"e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl\" (UID: \"993e8512-0a13-4565-8fd6-53d9f4910d95\") " pod="openshift-ingress/e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl"
Sep 09 16:29:57.485654 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:57.485618    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/993e8512-0a13-4565-8fd6-53d9f4910d95-istio-token\") pod \"e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl\" (UID: \"993e8512-0a13-4565-8fd6-53d9f4910d95\") " pod="openshift-ingress/e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl"
Sep 09 16:29:57.638998 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:57.638907    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl"
Sep 09 16:29:57.915170 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:57.915027    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl"]
Sep 09 16:29:57.917526 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:29:57.917496    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod993e8512_0a13_4565_8fd6_53d9f4910d95.slice/crio-6c9131879348bec23ca3d0e4d3282c5a22e76b99705f2bccecaaa557fdf26737 WatchSource:0}: Error finding container 6c9131879348bec23ca3d0e4d3282c5a22e76b99705f2bccecaaa557fdf26737: Status 404 returned error can't find the container with id 6c9131879348bec23ca3d0e4d3282c5a22e76b99705f2bccecaaa557fdf26737
Sep 09 16:29:57.919640 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:57.919604    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:29:57.919725 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:57.919671    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:29:57.919725 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:57.919700    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:29:58.626441 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:58.626402    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl" event={"ID":"993e8512-0a13-4565-8fd6-53d9f4910d95","Type":"ContainerStarted","Data":"0bbb8347070570f0310967a60f3d7c7cab59f2c4b33c3d7291097889e04def6c"}
Sep 09 16:29:58.626870 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:58.626487    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl" event={"ID":"993e8512-0a13-4565-8fd6-53d9f4910d95","Type":"ContainerStarted","Data":"6c9131879348bec23ca3d0e4d3282c5a22e76b99705f2bccecaaa557fdf26737"}
Sep 09 16:29:58.639549 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:58.639520    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl"
Sep 09 16:29:58.644780 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:58.644759    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl"
Sep 09 16:29:58.647288 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:58.647243    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl" podStartSLOduration=1.647228101 podStartE2EDuration="1.647228101s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 16:29:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 16:29:58.645180495 +0000 UTC m=+1274.063030494" watchObservedRunningTime="2026-09-09 16:29:58.647228101 +0000 UTC m=+1274.065078101"
Sep 09 16:29:59.631755 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:59.631720    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl"
Sep 09 16:29:59.632642 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:59.632617    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl"
Sep 09 16:29:59.690126 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:59.690091    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-13b02364-c4c4b69d8-b8hnv"]
Sep 09 16:29:59.690340 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:59.690306    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-13b02364-c4c4b69d8-b8hnv" podUID="d6aa3d61-5754-45bd-a397-0f997d1310bf" containerName="maas-api" containerID="cri-o://bc2abbebed5b42173171a5b718eac0aa1bb28ab8b04a2378db26172cff8b21a0" gracePeriod=30
Sep 09 16:29:59.941406 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:59.941382    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-13b02364-c4c4b69d8-b8hnv"
Sep 09 16:29:59.998200 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:59.998173    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6aa3d61-5754-45bd-a397-0f997d1310bf-openshift-service-ca\") pod \"d6aa3d61-5754-45bd-a397-0f997d1310bf\" (UID: \"d6aa3d61-5754-45bd-a397-0f997d1310bf\") "
Sep 09 16:29:59.998362 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:59.998222    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vxd7s\" (UniqueName: \"kubernetes.io/projected/d6aa3d61-5754-45bd-a397-0f997d1310bf-kube-api-access-vxd7s\") pod \"d6aa3d61-5754-45bd-a397-0f997d1310bf\" (UID: \"d6aa3d61-5754-45bd-a397-0f997d1310bf\") "
Sep 09 16:29:59.998412 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:59.998398    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6aa3d61-5754-45bd-a397-0f997d1310bf-maas-api-tls\") pod \"d6aa3d61-5754-45bd-a397-0f997d1310bf\" (UID: \"d6aa3d61-5754-45bd-a397-0f997d1310bf\") "
Sep 09 16:29:59.998550 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:59.998481    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6aa3d61-5754-45bd-a397-0f997d1310bf-metrics-certs\") pod \"d6aa3d61-5754-45bd-a397-0f997d1310bf\" (UID: \"d6aa3d61-5754-45bd-a397-0f997d1310bf\") "
Sep 09 16:29:59.998633 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:59.998568    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6aa3d61-5754-45bd-a397-0f997d1310bf-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d6aa3d61-5754-45bd-a397-0f997d1310bf" (UID: "d6aa3d61-5754-45bd-a397-0f997d1310bf"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 16:29:59.998777 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:29:59.998754    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6aa3d61-5754-45bd-a397-0f997d1310bf-openshift-service-ca\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:30:00.000692 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:00.000662    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d6aa3d61-5754-45bd-a397-0f997d1310bf-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d6aa3d61-5754-45bd-a397-0f997d1310bf" (UID: "d6aa3d61-5754-45bd-a397-0f997d1310bf"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 16:30:00.000773 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:00.000666    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6aa3d61-5754-45bd-a397-0f997d1310bf-kube-api-access-vxd7s" (OuterVolumeSpecName: "kube-api-access-vxd7s") pod "d6aa3d61-5754-45bd-a397-0f997d1310bf" (UID: "d6aa3d61-5754-45bd-a397-0f997d1310bf"). InnerVolumeSpecName "kube-api-access-vxd7s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 16:30:00.000773 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:00.000724    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d6aa3d61-5754-45bd-a397-0f997d1310bf-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "d6aa3d61-5754-45bd-a397-0f997d1310bf" (UID: "d6aa3d61-5754-45bd-a397-0f997d1310bf"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 16:30:00.099651 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:00.099620    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6aa3d61-5754-45bd-a397-0f997d1310bf-maas-api-tls\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:30:00.099651 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:00.099649    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6aa3d61-5754-45bd-a397-0f997d1310bf-metrics-certs\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:30:00.099651 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:00.099659    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vxd7s\" (UniqueName: \"kubernetes.io/projected/d6aa3d61-5754-45bd-a397-0f997d1310bf-kube-api-access-vxd7s\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:30:00.131957 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:00.131923    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29816190-kr87p"]
Sep 09 16:30:00.132357 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:00.132321    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6aa3d61-5754-45bd-a397-0f997d1310bf" containerName="maas-api"
Sep 09 16:30:00.132444 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:00.132360    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6aa3d61-5754-45bd-a397-0f997d1310bf" containerName="maas-api"
Sep 09 16:30:00.132535 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:00.132500    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6aa3d61-5754-45bd-a397-0f997d1310bf" containerName="maas-api"
Sep 09 16:30:00.135498 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:00.135450    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816190-kr87p"
Sep 09 16:30:00.137703 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:00.137686    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wxghs\""
Sep 09 16:30:00.143482 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:00.143446    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29816190-kr87p"]
Sep 09 16:30:00.147047 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:00.147029    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816190-kr87p"
Sep 09 16:30:00.421290 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:00.421262    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29816190-kr87p"]
Sep 09 16:30:00.423811 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:30:00.423786    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f8262a7_078f_450a_b5a8_5ee0480da094.slice/crio-3118db68f311b2a9386cefe20d96bdb433627254c6a0012a294a51eb297e4521 WatchSource:0}: Error finding container 3118db68f311b2a9386cefe20d96bdb433627254c6a0012a294a51eb297e4521: Status 404 returned error can't find the container with id 3118db68f311b2a9386cefe20d96bdb433627254c6a0012a294a51eb297e4521
Sep 09 16:30:00.637644 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:00.637603    2567 generic.go:356] "Generic (PLEG): container finished" podID="d6aa3d61-5754-45bd-a397-0f997d1310bf" containerID="bc2abbebed5b42173171a5b718eac0aa1bb28ab8b04a2378db26172cff8b21a0" exitCode=0
Sep 09 16:30:00.638063 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:00.637656    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-13b02364-c4c4b69d8-b8hnv"
Sep 09 16:30:00.638063 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:00.637687    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-13b02364-c4c4b69d8-b8hnv" event={"ID":"d6aa3d61-5754-45bd-a397-0f997d1310bf","Type":"ContainerDied","Data":"bc2abbebed5b42173171a5b718eac0aa1bb28ab8b04a2378db26172cff8b21a0"}
Sep 09 16:30:00.638063 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:00.637739    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-13b02364-c4c4b69d8-b8hnv" event={"ID":"d6aa3d61-5754-45bd-a397-0f997d1310bf","Type":"ContainerDied","Data":"adbed8b5d1983d92a626cabb5e702be6e43ae983eebf1dad94ce01c8da63737f"}
Sep 09 16:30:00.638063 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:00.637763    2567 scope.go:117] "RemoveContainer" containerID="bc2abbebed5b42173171a5b718eac0aa1bb28ab8b04a2378db26172cff8b21a0"
Sep 09 16:30:00.639427 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:00.639402    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816190-kr87p" event={"ID":"9f8262a7-078f-450a-b5a8-5ee0480da094","Type":"ContainerStarted","Data":"3118db68f311b2a9386cefe20d96bdb433627254c6a0012a294a51eb297e4521"}
Sep 09 16:30:00.647595 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:00.647577    2567 scope.go:117] "RemoveContainer" containerID="bc2abbebed5b42173171a5b718eac0aa1bb28ab8b04a2378db26172cff8b21a0"
Sep 09 16:30:00.647870 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:30:00.647853    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc2abbebed5b42173171a5b718eac0aa1bb28ab8b04a2378db26172cff8b21a0\": container with ID starting with bc2abbebed5b42173171a5b718eac0aa1bb28ab8b04a2378db26172cff8b21a0 not found: ID does not exist" containerID="bc2abbebed5b42173171a5b718eac0aa1bb28ab8b04a2378db26172cff8b21a0"
Sep 09 16:30:00.647916 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:00.647877    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc2abbebed5b42173171a5b718eac0aa1bb28ab8b04a2378db26172cff8b21a0"} err="failed to get container status \"bc2abbebed5b42173171a5b718eac0aa1bb28ab8b04a2378db26172cff8b21a0\": rpc error: code = NotFound desc = could not find container \"bc2abbebed5b42173171a5b718eac0aa1bb28ab8b04a2378db26172cff8b21a0\": container with ID starting with bc2abbebed5b42173171a5b718eac0aa1bb28ab8b04a2378db26172cff8b21a0 not found: ID does not exist"
Sep 09 16:30:00.658320 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:00.658293    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-13b02364-c4c4b69d8-b8hnv"]
Sep 09 16:30:00.666866 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:00.666843    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-13b02364-c4c4b69d8-b8hnv"]
Sep 09 16:30:01.042624 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:01.042586    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6aa3d61-5754-45bd-a397-0f997d1310bf" path="/var/lib/kubelet/pods/d6aa3d61-5754-45bd-a397-0f997d1310bf/volumes"
Sep 09 16:30:01.647374 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:01.647337    2567 generic.go:356] "Generic (PLEG): container finished" podID="9f8262a7-078f-450a-b5a8-5ee0480da094" containerID="63a865b2816b51928c4ea17da1ecb7d490a686c3976874fcd3415d9d80465bbe" exitCode=0
Sep 09 16:30:01.648149 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:01.647412    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816190-kr87p" event={"ID":"9f8262a7-078f-450a-b5a8-5ee0480da094","Type":"ContainerDied","Data":"63a865b2816b51928c4ea17da1ecb7d490a686c3976874fcd3415d9d80465bbe"}
Sep 09 16:30:01.648149 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:01.647647    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9520a4-cb5556ffd-ndqxc"]
Sep 09 16:30:01.651691 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:01.651666    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9520a4-cb5556ffd-ndqxc"
Sep 09 16:30:01.654015 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:01.653992    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-9520a4-serving-cert\""
Sep 09 16:30:01.654716 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:01.654695    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-mslnv\""
Sep 09 16:30:01.654830 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:01.654696    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 09 16:30:01.654830 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:01.654696    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 09 16:30:01.654830 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:01.654696    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 09 16:30:01.663571 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:01.663331    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9520a4-cb5556ffd-ndqxc"]
Sep 09 16:30:01.714767 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:01.714730    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e9ca9ce-194e-4c2b-a02a-7b514ecf6385-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-9520a4-cb5556ffd-ndqxc\" (UID: \"5e9ca9ce-194e-4c2b-a02a-7b514ecf6385\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9520a4-cb5556ffd-ndqxc"
Sep 09 16:30:01.714977 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:01.714783    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5e9ca9ce-194e-4c2b-a02a-7b514ecf6385-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-9520a4-cb5556ffd-ndqxc\" (UID: \"5e9ca9ce-194e-4c2b-a02a-7b514ecf6385\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9520a4-cb5556ffd-ndqxc"
Sep 09 16:30:01.714977 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:01.714808    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e9ca9ce-194e-4c2b-a02a-7b514ecf6385-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-9520a4-cb5556ffd-ndqxc\" (UID: \"5e9ca9ce-194e-4c2b-a02a-7b514ecf6385\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9520a4-cb5556ffd-ndqxc"
Sep 09 16:30:01.714977 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:01.714857    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9llbm\" (UniqueName: \"kubernetes.io/projected/5e9ca9ce-194e-4c2b-a02a-7b514ecf6385-kube-api-access-9llbm\") pod \"maas-api-e2e-shared-b-w3-9520a4-cb5556ffd-ndqxc\" (UID: \"5e9ca9ce-194e-4c2b-a02a-7b514ecf6385\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9520a4-cb5556ffd-ndqxc"
Sep 09 16:30:01.815868 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:01.815826    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5e9ca9ce-194e-4c2b-a02a-7b514ecf6385-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-9520a4-cb5556ffd-ndqxc\" (UID: \"5e9ca9ce-194e-4c2b-a02a-7b514ecf6385\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9520a4-cb5556ffd-ndqxc"
Sep 09 16:30:01.816071 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:01.815880    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e9ca9ce-194e-4c2b-a02a-7b514ecf6385-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-9520a4-cb5556ffd-ndqxc\" (UID: \"5e9ca9ce-194e-4c2b-a02a-7b514ecf6385\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9520a4-cb5556ffd-ndqxc"
Sep 09 16:30:01.816071 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:01.815921    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9llbm\" (UniqueName: \"kubernetes.io/projected/5e9ca9ce-194e-4c2b-a02a-7b514ecf6385-kube-api-access-9llbm\") pod \"maas-api-e2e-shared-b-w3-9520a4-cb5556ffd-ndqxc\" (UID: \"5e9ca9ce-194e-4c2b-a02a-7b514ecf6385\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9520a4-cb5556ffd-ndqxc"
Sep 09 16:30:01.816071 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:30:01.815954    2567 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9520a4-serving-cert: secret "maas-api-e2e-shared-b-w3-9520a4-serving-cert" not found
Sep 09 16:30:01.816071 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:01.815992    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e9ca9ce-194e-4c2b-a02a-7b514ecf6385-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-9520a4-cb5556ffd-ndqxc\" (UID: \"5e9ca9ce-194e-4c2b-a02a-7b514ecf6385\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9520a4-cb5556ffd-ndqxc"
Sep 09 16:30:01.816071 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:30:01.816017    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e9ca9ce-194e-4c2b-a02a-7b514ecf6385-maas-api-tls podName:5e9ca9ce-194e-4c2b-a02a-7b514ecf6385 nodeName:}" failed. No retries permitted until 2026-09-09 16:30:02.316000906 +0000 UTC m=+1277.733850887 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5e9ca9ce-194e-4c2b-a02a-7b514ecf6385-maas-api-tls") pod "maas-api-e2e-shared-b-w3-9520a4-cb5556ffd-ndqxc" (UID: "5e9ca9ce-194e-4c2b-a02a-7b514ecf6385") : secret "maas-api-e2e-shared-b-w3-9520a4-serving-cert" not found
Sep 09 16:30:01.816762 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:01.816738    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e9ca9ce-194e-4c2b-a02a-7b514ecf6385-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-9520a4-cb5556ffd-ndqxc\" (UID: \"5e9ca9ce-194e-4c2b-a02a-7b514ecf6385\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9520a4-cb5556ffd-ndqxc"
Sep 09 16:30:01.818713 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:01.818694    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e9ca9ce-194e-4c2b-a02a-7b514ecf6385-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-9520a4-cb5556ffd-ndqxc\" (UID: \"5e9ca9ce-194e-4c2b-a02a-7b514ecf6385\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9520a4-cb5556ffd-ndqxc"
Sep 09 16:30:01.824166 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:01.824142    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9llbm\" (UniqueName: \"kubernetes.io/projected/5e9ca9ce-194e-4c2b-a02a-7b514ecf6385-kube-api-access-9llbm\") pod \"maas-api-e2e-shared-b-w3-9520a4-cb5556ffd-ndqxc\" (UID: \"5e9ca9ce-194e-4c2b-a02a-7b514ecf6385\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9520a4-cb5556ffd-ndqxc"
Sep 09 16:30:02.320321 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:02.320283    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5e9ca9ce-194e-4c2b-a02a-7b514ecf6385-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-9520a4-cb5556ffd-ndqxc\" (UID: \"5e9ca9ce-194e-4c2b-a02a-7b514ecf6385\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9520a4-cb5556ffd-ndqxc"
Sep 09 16:30:02.323272 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:02.323224    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5e9ca9ce-194e-4c2b-a02a-7b514ecf6385-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-9520a4-cb5556ffd-ndqxc\" (UID: \"5e9ca9ce-194e-4c2b-a02a-7b514ecf6385\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9520a4-cb5556ffd-ndqxc"
Sep 09 16:30:02.566284 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:02.566251    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9520a4-cb5556ffd-ndqxc"
Sep 09 16:30:02.777961 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:02.777936    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816190-kr87p"
Sep 09 16:30:02.836055 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:02.836031    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9520a4-cb5556ffd-ndqxc"]
Sep 09 16:30:02.838182 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:30:02.838155    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e9ca9ce_194e_4c2b_a02a_7b514ecf6385.slice/crio-39acbd085c49da144ca113a0897cffead52ab65fa57da801318672a25e73cfbb WatchSource:0}: Error finding container 39acbd085c49da144ca113a0897cffead52ab65fa57da801318672a25e73cfbb: Status 404 returned error can't find the container with id 39acbd085c49da144ca113a0897cffead52ab65fa57da801318672a25e73cfbb
Sep 09 16:30:03.662682 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:03.662645    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9520a4-cb5556ffd-ndqxc" event={"ID":"5e9ca9ce-194e-4c2b-a02a-7b514ecf6385","Type":"ContainerStarted","Data":"39acbd085c49da144ca113a0897cffead52ab65fa57da801318672a25e73cfbb"}
Sep 09 16:30:03.664032 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:03.664012    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816190-kr87p"
Sep 09 16:30:03.664032 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:03.664022    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816190-kr87p" event={"ID":"9f8262a7-078f-450a-b5a8-5ee0480da094","Type":"ContainerDied","Data":"3118db68f311b2a9386cefe20d96bdb433627254c6a0012a294a51eb297e4521"}
Sep 09 16:30:03.664190 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:03.664055    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3118db68f311b2a9386cefe20d96bdb433627254c6a0012a294a51eb297e4521"
Sep 09 16:30:04.670668 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:04.670631    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9520a4-cb5556ffd-ndqxc" event={"ID":"5e9ca9ce-194e-4c2b-a02a-7b514ecf6385","Type":"ContainerStarted","Data":"db24911dbe2115b81d0c6770e95b3d9c2568978fb8c8455d3f654075515b9a6a"}
Sep 09 16:30:04.671085 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:04.670695    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9520a4-cb5556ffd-ndqxc"
Sep 09 16:30:04.686413 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:04.686358    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9520a4-cb5556ffd-ndqxc" podStartSLOduration=2.6112793549999997 podStartE2EDuration="3.686338172s" totalImagesPullingTime="1.075058817s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 16:30:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 16:30:04.686290516 +0000 UTC m=+1280.104140530" watchObservedRunningTime="2026-09-09 16:30:04.686338172 +0000 UTC m=+1280.104188168"
Sep 09 16:30:07.562958 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:07.562919    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:30:10.682026 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:10.681988    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9520a4-cb5556ffd-ndqxc"
Sep 09 16:30:15.673645 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:15.673604    2567 patch_prober.go:28] interesting pod/e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 09 16:30:15.674042 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:15.673673    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl" podUID="993e8512-0a13-4565-8fd6-53d9f4910d95" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 09 16:30:17.750571 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:17.750540    2567 generic.go:356] "Generic (PLEG): container finished" podID="993e8512-0a13-4565-8fd6-53d9f4910d95" containerID="0bbb8347070570f0310967a60f3d7c7cab59f2c4b33c3d7291097889e04def6c" exitCode=0
Sep 09 16:30:17.751001 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:17.750624    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl" event={"ID":"993e8512-0a13-4565-8fd6-53d9f4910d95","Type":"ContainerDied","Data":"0bbb8347070570f0310967a60f3d7c7cab59f2c4b33c3d7291097889e04def6c"}
Sep 09 16:30:17.751043 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:17.751029    2567 scope.go:117] "RemoveContainer" containerID="0bbb8347070570f0310967a60f3d7c7cab59f2c4b33c3d7291097889e04def6c"
Sep 09 16:30:17.752079 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:17.752049    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:30:17.752146 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:17.752133    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:30:17.752193 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:17.752169    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:30:18.758125 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:18.758094    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl" event={"ID":"993e8512-0a13-4565-8fd6-53d9f4910d95","Type":"ContainerStarted","Data":"87307eb8f599b4d36bfe7390cf78211a77ed899f0c89e046572a39954a37acb3"}
Sep 09 16:30:19.639254 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:19.639212    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl"
Sep 09 16:30:19.639438 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:19.639387    2567 patch_prober.go:28] interesting pod/e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Sep 09 16:30:19.639508 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:19.639435    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl" podUID="993e8512-0a13-4565-8fd6-53d9f4910d95" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Sep 09 16:30:20.026852 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:20.026769    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4"]
Sep 09 16:30:20.027193 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:20.027147    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f8262a7-078f-450a-b5a8-5ee0480da094" containerName="cleanup"
Sep 09 16:30:20.027193 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:20.027158    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f8262a7-078f-450a-b5a8-5ee0480da094" containerName="cleanup"
Sep 09 16:30:20.027267 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:20.027232    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f8262a7-078f-450a-b5a8-5ee0480da094" containerName="cleanup"
Sep 09 16:30:20.030664 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:20.030638    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4"
Sep 09 16:30:20.033004 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:20.032977    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-042f8b-openshift-default-dockercfg-wvr6d\""
Sep 09 16:30:20.047286 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:20.047260    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4"]
Sep 09 16:30:20.097983 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:20.097936    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/de55f322-3dd3-4c96-967f-c99336cad8ac-workload-certs\") pod \"e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4\" (UID: \"de55f322-3dd3-4c96-967f-c99336cad8ac\") " pod="openshift-ingress/e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4"
Sep 09 16:30:20.098263 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:20.098238    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/de55f322-3dd3-4c96-967f-c99336cad8ac-istio-data\") pod \"e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4\" (UID: \"de55f322-3dd3-4c96-967f-c99336cad8ac\") " pod="openshift-ingress/e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4"
Sep 09 16:30:20.098412 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:20.098393    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8tps\" (UniqueName: \"kubernetes.io/projected/de55f322-3dd3-4c96-967f-c99336cad8ac-kube-api-access-q8tps\") pod \"e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4\" (UID: \"de55f322-3dd3-4c96-967f-c99336cad8ac\") " pod="openshift-ingress/e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4"
Sep 09 16:30:20.098623 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:20.098595    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/de55f322-3dd3-4c96-967f-c99336cad8ac-workload-socket\") pod \"e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4\" (UID: \"de55f322-3dd3-4c96-967f-c99336cad8ac\") " pod="openshift-ingress/e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4"
Sep 09 16:30:20.098829 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:20.098809    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/de55f322-3dd3-4c96-967f-c99336cad8ac-credential-socket\") pod \"e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4\" (UID: \"de55f322-3dd3-4c96-967f-c99336cad8ac\") " pod="openshift-ingress/e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4"
Sep 09 16:30:20.098953 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:20.098938    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/de55f322-3dd3-4c96-967f-c99336cad8ac-istio-podinfo\") pod \"e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4\" (UID: \"de55f322-3dd3-4c96-967f-c99336cad8ac\") " pod="openshift-ingress/e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4"
Sep 09 16:30:20.099087 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:20.099071    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de55f322-3dd3-4c96-967f-c99336cad8ac-istio-token\") pod \"e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4\" (UID: \"de55f322-3dd3-4c96-967f-c99336cad8ac\") " pod="openshift-ingress/e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4"
Sep 09 16:30:20.099213 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:20.099200    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/de55f322-3dd3-4c96-967f-c99336cad8ac-istio-envoy\") pod \"e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4\" (UID: \"de55f322-3dd3-4c96-967f-c99336cad8ac\") " pod="openshift-ingress/e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4"
Sep 09 16:30:20.099314 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:20.099300    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/de55f322-3dd3-4c96-967f-c99336cad8ac-istiod-ca-cert\") pod \"e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4\" (UID: \"de55f322-3dd3-4c96-967f-c99336cad8ac\") " pod="openshift-ingress/e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4"
Sep 09 16:30:20.201116 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:20.201069    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/de55f322-3dd3-4c96-967f-c99336cad8ac-credential-socket\") pod \"e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4\" (UID: \"de55f322-3dd3-4c96-967f-c99336cad8ac\") " pod="openshift-ingress/e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4"
Sep 09 16:30:20.201324 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:20.201135    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/de55f322-3dd3-4c96-967f-c99336cad8ac-istio-podinfo\") pod \"e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4\" (UID: \"de55f322-3dd3-4c96-967f-c99336cad8ac\") " pod="openshift-ingress/e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4"
Sep 09 16:30:20.201324 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:20.201185    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de55f322-3dd3-4c96-967f-c99336cad8ac-istio-token\") pod \"e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4\" (UID: \"de55f322-3dd3-4c96-967f-c99336cad8ac\") " pod="openshift-ingress/e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4"
Sep 09 16:30:20.201324 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:20.201226    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/de55f322-3dd3-4c96-967f-c99336cad8ac-istio-envoy\") pod \"e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4\" (UID: \"de55f322-3dd3-4c96-967f-c99336cad8ac\") " pod="openshift-ingress/e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4"
Sep 09 16:30:20.201324 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:20.201252    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/de55f322-3dd3-4c96-967f-c99336cad8ac-istiod-ca-cert\") pod \"e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4\" (UID: \"de55f322-3dd3-4c96-967f-c99336cad8ac\") " pod="openshift-ingress/e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4"
Sep 09 16:30:20.201324 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:20.201319    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/de55f322-3dd3-4c96-967f-c99336cad8ac-workload-certs\") pod \"e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4\" (UID: \"de55f322-3dd3-4c96-967f-c99336cad8ac\") " pod="openshift-ingress/e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4"
Sep 09 16:30:20.201678 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:20.201350    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/de55f322-3dd3-4c96-967f-c99336cad8ac-istio-data\") pod \"e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4\" (UID: \"de55f322-3dd3-4c96-967f-c99336cad8ac\") " pod="openshift-ingress/e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4"
Sep 09 16:30:20.201678 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:20.201385    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8tps\" (UniqueName: \"kubernetes.io/projected/de55f322-3dd3-4c96-967f-c99336cad8ac-kube-api-access-q8tps\") pod \"e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4\" (UID: \"de55f322-3dd3-4c96-967f-c99336cad8ac\") " pod="openshift-ingress/e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4"
Sep 09 16:30:20.201678 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:20.201434    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/de55f322-3dd3-4c96-967f-c99336cad8ac-workload-socket\") pod \"e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4\" (UID: \"de55f322-3dd3-4c96-967f-c99336cad8ac\") " pod="openshift-ingress/e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4"
Sep 09 16:30:20.201678 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:20.201553    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/de55f322-3dd3-4c96-967f-c99336cad8ac-credential-socket\") pod \"e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4\" (UID: \"de55f322-3dd3-4c96-967f-c99336cad8ac\") " pod="openshift-ingress/e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4"
Sep 09 16:30:20.201900 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:20.201763    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/de55f322-3dd3-4c96-967f-c99336cad8ac-workload-socket\") pod \"e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4\" (UID: \"de55f322-3dd3-4c96-967f-c99336cad8ac\") " pod="openshift-ingress/e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4"
Sep 09 16:30:20.202006 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:20.201982    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/de55f322-3dd3-4c96-967f-c99336cad8ac-workload-certs\") pod \"e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4\" (UID: \"de55f322-3dd3-4c96-967f-c99336cad8ac\") " pod="openshift-ingress/e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4"
Sep 09 16:30:20.202203 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:20.202178    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/de55f322-3dd3-4c96-967f-c99336cad8ac-istiod-ca-cert\") pod \"e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4\" (UID: \"de55f322-3dd3-4c96-967f-c99336cad8ac\") " pod="openshift-ingress/e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4"
Sep 09 16:30:20.202288 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:20.202215    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/de55f322-3dd3-4c96-967f-c99336cad8ac-istio-data\") pod \"e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4\" (UID: \"de55f322-3dd3-4c96-967f-c99336cad8ac\") " pod="openshift-ingress/e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4"
Sep 09 16:30:20.207048 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:20.207013    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/de55f322-3dd3-4c96-967f-c99336cad8ac-istio-podinfo\") pod \"e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4\" (UID: \"de55f322-3dd3-4c96-967f-c99336cad8ac\") " pod="openshift-ingress/e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4"
Sep 09 16:30:20.212545 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:20.209076    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/de55f322-3dd3-4c96-967f-c99336cad8ac-istio-envoy\") pod \"e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4\" (UID: \"de55f322-3dd3-4c96-967f-c99336cad8ac\") " pod="openshift-ingress/e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4"
Sep 09 16:30:20.212977 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:20.212950    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de55f322-3dd3-4c96-967f-c99336cad8ac-istio-token\") pod \"e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4\" (UID: \"de55f322-3dd3-4c96-967f-c99336cad8ac\") " pod="openshift-ingress/e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4"
Sep 09 16:30:20.218192 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:20.218166    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8tps\" (UniqueName: \"kubernetes.io/projected/de55f322-3dd3-4c96-967f-c99336cad8ac-kube-api-access-q8tps\") pod \"e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4\" (UID: \"de55f322-3dd3-4c96-967f-c99336cad8ac\") " pod="openshift-ingress/e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4"
Sep 09 16:30:20.342574 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:20.342528    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4"
Sep 09 16:30:20.639561 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:20.639429    2567 patch_prober.go:28] interesting pod/e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Sep 09 16:30:20.639561 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:20.639511    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl" podUID="993e8512-0a13-4565-8fd6-53d9f4910d95" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Sep 09 16:30:20.640911 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:20.640882    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4"]
Sep 09 16:30:20.643340 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:30:20.643316    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde55f322_3dd3_4c96_967f_c99336cad8ac.slice/crio-afdfcc0f07249259a7a788d6f5cf9f84ccb412331a95f17f040b3a054645e792 WatchSource:0}: Error finding container afdfcc0f07249259a7a788d6f5cf9f84ccb412331a95f17f040b3a054645e792: Status 404 returned error can't find the container with id afdfcc0f07249259a7a788d6f5cf9f84ccb412331a95f17f040b3a054645e792
Sep 09 16:30:20.645628 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:20.645599    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:30:20.645690 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:20.645663    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:30:20.645730 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:20.645697    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:30:20.771441 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:20.771411    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4" event={"ID":"de55f322-3dd3-4c96-967f-c99336cad8ac","Type":"ContainerStarted","Data":"8e02c81b37ff3bb3fe0e7a2ab9b2887e90bfd7225a0c892ce632d137e8d09099"}
Sep 09 16:30:20.771626 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:20.771485    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4" event={"ID":"de55f322-3dd3-4c96-967f-c99336cad8ac","Type":"ContainerStarted","Data":"afdfcc0f07249259a7a788d6f5cf9f84ccb412331a95f17f040b3a054645e792"}
Sep 09 16:30:20.792542 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:20.792480    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4" podStartSLOduration=0.792440694 podStartE2EDuration="792.440694ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 16:30:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 16:30:20.789586699 +0000 UTC m=+1296.207436699" watchObservedRunningTime="2026-09-09 16:30:20.792440694 +0000 UTC m=+1296.210290697"
Sep 09 16:30:21.343034 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:21.342991    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4"
Sep 09 16:30:21.348589 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:21.348564    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4"
Sep 09 16:30:21.639646 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:21.639558    2567 patch_prober.go:28] interesting pod/e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Sep 09 16:30:21.639646 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:21.639617    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl" podUID="993e8512-0a13-4565-8fd6-53d9f4910d95" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Sep 09 16:30:21.776123 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:21.776092    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4"
Sep 09 16:30:21.777013 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:21.776997    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4"
Sep 09 16:30:22.640324 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:22.640284    2567 patch_prober.go:28] interesting pod/e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Sep 09 16:30:22.640763 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:22.640345    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl" podUID="993e8512-0a13-4565-8fd6-53d9f4910d95" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Sep 09 16:30:23.639930 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:23.639885    2567 patch_prober.go:28] interesting pod/e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Sep 09 16:30:23.640109 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:23.639947    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl" podUID="993e8512-0a13-4565-8fd6-53d9f4910d95" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Sep 09 16:30:24.643345 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:24.643314    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl"
Sep 09 16:30:24.643767 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:24.643672    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl"
Sep 09 16:30:24.644278 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:24.644263    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl"
Sep 09 16:30:28.612726 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:28.612687    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:30:37.817649 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:37.817595    2567 patch_prober.go:28] interesting pod/e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 09 16:30:37.818182 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:37.817678    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4" podUID="de55f322-3dd3-4c96-967f-c99336cad8ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 09 16:30:39.873550 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:39.873511    2567 generic.go:356] "Generic (PLEG): container finished" podID="de55f322-3dd3-4c96-967f-c99336cad8ac" containerID="8e02c81b37ff3bb3fe0e7a2ab9b2887e90bfd7225a0c892ce632d137e8d09099" exitCode=0
Sep 09 16:30:39.873966 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:39.873589    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4" event={"ID":"de55f322-3dd3-4c96-967f-c99336cad8ac","Type":"ContainerDied","Data":"8e02c81b37ff3bb3fe0e7a2ab9b2887e90bfd7225a0c892ce632d137e8d09099"}
Sep 09 16:30:39.874019 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:39.874003    2567 scope.go:117] "RemoveContainer" containerID="8e02c81b37ff3bb3fe0e7a2ab9b2887e90bfd7225a0c892ce632d137e8d09099"
Sep 09 16:30:39.875031 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:39.875003    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:30:39.875086 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:39.875070    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:30:39.875124 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:39.875103    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:30:40.881034 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:40.880997    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4" event={"ID":"de55f322-3dd3-4c96-967f-c99336cad8ac","Type":"ContainerStarted","Data":"04ef0ceb5b970451d72a976bc2b53827b2041bcc97498a2caadbce269b0e5f1c"}
Sep 09 16:30:41.342765 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:41.342729    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4"
Sep 09 16:30:41.343021 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:41.342998    2567 patch_prober.go:28] interesting pod/e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Sep 09 16:30:41.343112 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:41.343057    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4" podUID="de55f322-3dd3-4c96-967f-c99336cad8ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Sep 09 16:30:42.343042 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:42.343006    2567 patch_prober.go:28] interesting pod/e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Sep 09 16:30:42.343684 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:42.343076    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4" podUID="de55f322-3dd3-4c96-967f-c99336cad8ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Sep 09 16:30:43.343294 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:43.343254    2567 patch_prober.go:28] interesting pod/e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Sep 09 16:30:43.343718 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:43.343313    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4" podUID="de55f322-3dd3-4c96-967f-c99336cad8ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Sep 09 16:30:44.342954 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:44.342913    2567 patch_prober.go:28] interesting pod/e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Sep 09 16:30:44.343156 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:44.342969    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4" podUID="de55f322-3dd3-4c96-967f-c99336cad8ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Sep 09 16:30:45.343083 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:45.343048    2567 patch_prober.go:28] interesting pod/e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Sep 09 16:30:45.343523 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:45.343103    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4" podUID="de55f322-3dd3-4c96-967f-c99336cad8ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Sep 09 16:30:46.346806 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:46.346774    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4"
Sep 09 16:30:46.347186 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:46.347042    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4"
Sep 09 16:30:46.347770 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:46.347746    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4"
Sep 09 16:30:48.359380 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:48.359352    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m"]
Sep 09 16:30:48.364441 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:48.364419    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m"
Sep 09 16:30:48.367059 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:48.367037    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-de97ae-openshift-default-dockercfg-47fdj\""
Sep 09 16:30:48.374888 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:48.374866    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m"]
Sep 09 16:30:48.483226 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:48.483189    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phfdt\" (UniqueName: \"kubernetes.io/projected/5c55fb39-101d-4ba1-9f9d-ff6a97232826-kube-api-access-phfdt\") pod \"e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m\" (UID: \"5c55fb39-101d-4ba1-9f9d-ff6a97232826\") " pod="openshift-ingress/e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m"
Sep 09 16:30:48.483564 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:48.483525    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c55fb39-101d-4ba1-9f9d-ff6a97232826-workload-socket\") pod \"e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m\" (UID: \"5c55fb39-101d-4ba1-9f9d-ff6a97232826\") " pod="openshift-ingress/e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m"
Sep 09 16:30:48.483703 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:48.483586    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c55fb39-101d-4ba1-9f9d-ff6a97232826-istio-podinfo\") pod \"e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m\" (UID: \"5c55fb39-101d-4ba1-9f9d-ff6a97232826\") " pod="openshift-ingress/e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m"
Sep 09 16:30:48.483703 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:48.483640    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c55fb39-101d-4ba1-9f9d-ff6a97232826-istio-token\") pod \"e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m\" (UID: \"5c55fb39-101d-4ba1-9f9d-ff6a97232826\") " pod="openshift-ingress/e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m"
Sep 09 16:30:48.483703 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:48.483671    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c55fb39-101d-4ba1-9f9d-ff6a97232826-istio-data\") pod \"e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m\" (UID: \"5c55fb39-101d-4ba1-9f9d-ff6a97232826\") " pod="openshift-ingress/e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m"
Sep 09 16:30:48.483864 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:48.483708    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c55fb39-101d-4ba1-9f9d-ff6a97232826-credential-socket\") pod \"e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m\" (UID: \"5c55fb39-101d-4ba1-9f9d-ff6a97232826\") " pod="openshift-ingress/e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m"
Sep 09 16:30:48.483864 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:48.483735    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c55fb39-101d-4ba1-9f9d-ff6a97232826-workload-certs\") pod \"e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m\" (UID: \"5c55fb39-101d-4ba1-9f9d-ff6a97232826\") " pod="openshift-ingress/e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m"
Sep 09 16:30:48.483864 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:48.483779    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c55fb39-101d-4ba1-9f9d-ff6a97232826-istio-envoy\") pod \"e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m\" (UID: \"5c55fb39-101d-4ba1-9f9d-ff6a97232826\") " pod="openshift-ingress/e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m"
Sep 09 16:30:48.483864 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:48.483833    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c55fb39-101d-4ba1-9f9d-ff6a97232826-istiod-ca-cert\") pod \"e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m\" (UID: \"5c55fb39-101d-4ba1-9f9d-ff6a97232826\") " pod="openshift-ingress/e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m"
Sep 09 16:30:48.584650 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:48.584614    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c55fb39-101d-4ba1-9f9d-ff6a97232826-credential-socket\") pod \"e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m\" (UID: \"5c55fb39-101d-4ba1-9f9d-ff6a97232826\") " pod="openshift-ingress/e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m"
Sep 09 16:30:48.584650 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:48.584652    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c55fb39-101d-4ba1-9f9d-ff6a97232826-workload-certs\") pod \"e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m\" (UID: \"5c55fb39-101d-4ba1-9f9d-ff6a97232826\") " pod="openshift-ingress/e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m"
Sep 09 16:30:48.584905 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:48.584684    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c55fb39-101d-4ba1-9f9d-ff6a97232826-istio-envoy\") pod \"e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m\" (UID: \"5c55fb39-101d-4ba1-9f9d-ff6a97232826\") " pod="openshift-ingress/e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m"
Sep 09 16:30:48.584905 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:48.584760    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c55fb39-101d-4ba1-9f9d-ff6a97232826-istiod-ca-cert\") pod \"e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m\" (UID: \"5c55fb39-101d-4ba1-9f9d-ff6a97232826\") " pod="openshift-ingress/e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m"
Sep 09 16:30:48.584905 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:48.584800    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phfdt\" (UniqueName: \"kubernetes.io/projected/5c55fb39-101d-4ba1-9f9d-ff6a97232826-kube-api-access-phfdt\") pod \"e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m\" (UID: \"5c55fb39-101d-4ba1-9f9d-ff6a97232826\") " pod="openshift-ingress/e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m"
Sep 09 16:30:48.584905 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:48.584846    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c55fb39-101d-4ba1-9f9d-ff6a97232826-workload-socket\") pod \"e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m\" (UID: \"5c55fb39-101d-4ba1-9f9d-ff6a97232826\") " pod="openshift-ingress/e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m"
Sep 09 16:30:48.585111 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:48.584987    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c55fb39-101d-4ba1-9f9d-ff6a97232826-istio-podinfo\") pod \"e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m\" (UID: \"5c55fb39-101d-4ba1-9f9d-ff6a97232826\") " pod="openshift-ingress/e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m"
Sep 09 16:30:48.585111 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:48.585067    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c55fb39-101d-4ba1-9f9d-ff6a97232826-istio-token\") pod \"e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m\" (UID: \"5c55fb39-101d-4ba1-9f9d-ff6a97232826\") " pod="openshift-ingress/e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m"
Sep 09 16:30:48.585111 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:48.585103    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c55fb39-101d-4ba1-9f9d-ff6a97232826-istio-data\") pod \"e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m\" (UID: \"5c55fb39-101d-4ba1-9f9d-ff6a97232826\") " pod="openshift-ingress/e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m"
Sep 09 16:30:48.585258 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:48.585120    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c55fb39-101d-4ba1-9f9d-ff6a97232826-workload-certs\") pod \"e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m\" (UID: \"5c55fb39-101d-4ba1-9f9d-ff6a97232826\") " pod="openshift-ingress/e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m"
Sep 09 16:30:48.585258 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:48.585066    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c55fb39-101d-4ba1-9f9d-ff6a97232826-credential-socket\") pod \"e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m\" (UID: \"5c55fb39-101d-4ba1-9f9d-ff6a97232826\") " pod="openshift-ingress/e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m"
Sep 09 16:30:48.585258 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:48.585171    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c55fb39-101d-4ba1-9f9d-ff6a97232826-workload-socket\") pod \"e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m\" (UID: \"5c55fb39-101d-4ba1-9f9d-ff6a97232826\") " pod="openshift-ingress/e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m"
Sep 09 16:30:48.585553 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:48.585532    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c55fb39-101d-4ba1-9f9d-ff6a97232826-istiod-ca-cert\") pod \"e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m\" (UID: \"5c55fb39-101d-4ba1-9f9d-ff6a97232826\") " pod="openshift-ingress/e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m"
Sep 09 16:30:48.585673 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:48.585559    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c55fb39-101d-4ba1-9f9d-ff6a97232826-istio-data\") pod \"e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m\" (UID: \"5c55fb39-101d-4ba1-9f9d-ff6a97232826\") " pod="openshift-ingress/e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m"
Sep 09 16:30:48.587288 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:48.587270    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c55fb39-101d-4ba1-9f9d-ff6a97232826-istio-envoy\") pod \"e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m\" (UID: \"5c55fb39-101d-4ba1-9f9d-ff6a97232826\") " pod="openshift-ingress/e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m"
Sep 09 16:30:48.587649 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:48.587630    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c55fb39-101d-4ba1-9f9d-ff6a97232826-istio-podinfo\") pod \"e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m\" (UID: \"5c55fb39-101d-4ba1-9f9d-ff6a97232826\") " pod="openshift-ingress/e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m"
Sep 09 16:30:48.592254 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:48.592226    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c55fb39-101d-4ba1-9f9d-ff6a97232826-istio-token\") pod \"e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m\" (UID: \"5c55fb39-101d-4ba1-9f9d-ff6a97232826\") " pod="openshift-ingress/e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m"
Sep 09 16:30:48.592421 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:48.592403    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-phfdt\" (UniqueName: \"kubernetes.io/projected/5c55fb39-101d-4ba1-9f9d-ff6a97232826-kube-api-access-phfdt\") pod \"e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m\" (UID: \"5c55fb39-101d-4ba1-9f9d-ff6a97232826\") " pod="openshift-ingress/e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m"
Sep 09 16:30:48.676425 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:48.676338    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m"
Sep 09 16:30:48.952612 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:48.952584    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m"]
Sep 09 16:30:48.954575 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:30:48.954547    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c55fb39_101d_4ba1_9f9d_ff6a97232826.slice/crio-4cd4d7a6cc12d92c20042a4af0d05177d0afee2301a03a7f91c30965ba997d29 WatchSource:0}: Error finding container 4cd4d7a6cc12d92c20042a4af0d05177d0afee2301a03a7f91c30965ba997d29: Status 404 returned error can't find the container with id 4cd4d7a6cc12d92c20042a4af0d05177d0afee2301a03a7f91c30965ba997d29
Sep 09 16:30:48.956958 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:48.956929    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:30:48.957027 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:48.956995    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:30:48.957067 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:48.957025    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:30:49.945855 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:49.945816    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m" event={"ID":"5c55fb39-101d-4ba1-9f9d-ff6a97232826","Type":"ContainerStarted","Data":"1948df54257d60064fa43c7205e1afaa325b400477ed0c89a204244d06ed37c1"}
Sep 09 16:30:49.946238 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:49.945902    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m" event={"ID":"5c55fb39-101d-4ba1-9f9d-ff6a97232826","Type":"ContainerStarted","Data":"4cd4d7a6cc12d92c20042a4af0d05177d0afee2301a03a7f91c30965ba997d29"}
Sep 09 16:30:49.968521 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:49.968436    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m" podStartSLOduration=1.968419674 podStartE2EDuration="1.968419674s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 16:30:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 16:30:49.966687851 +0000 UTC m=+1325.384537851" watchObservedRunningTime="2026-09-09 16:30:49.968419674 +0000 UTC m=+1325.386269752"
Sep 09 16:30:50.676839 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:50.676805    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m"
Sep 09 16:30:50.682194 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:50.682164    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m"
Sep 09 16:30:50.950946 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:50.950867    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m"
Sep 09 16:30:50.951870 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:50.951853    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m"
Sep 09 16:30:51.319429 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:51.319397    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:30:53.044561 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:53.044517    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-de97ae-558ddff7f7-zh2h7"]
Sep 09 16:30:53.049488 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:53.049444    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-de97ae-558ddff7f7-zh2h7"
Sep 09 16:30:53.051757 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:53.051723    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-de97ae-serving-cert\""
Sep 09 16:30:53.058513 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:53.058469    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-de97ae-558ddff7f7-zh2h7"]
Sep 09 16:30:53.129898 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:53.129855    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgprf\" (UniqueName: \"kubernetes.io/projected/6a2a3589-419e-47ba-bc4a-bbc4ddc9ff3e-kube-api-access-xgprf\") pod \"maas-api-e2e-shared-b-w2-de97ae-558ddff7f7-zh2h7\" (UID: \"6a2a3589-419e-47ba-bc4a-bbc4ddc9ff3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-de97ae-558ddff7f7-zh2h7"
Sep 09 16:30:53.129898 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:53.129898    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6a2a3589-419e-47ba-bc4a-bbc4ddc9ff3e-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-de97ae-558ddff7f7-zh2h7\" (UID: \"6a2a3589-419e-47ba-bc4a-bbc4ddc9ff3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-de97ae-558ddff7f7-zh2h7"
Sep 09 16:30:53.130133 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:53.129926    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6a2a3589-419e-47ba-bc4a-bbc4ddc9ff3e-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-de97ae-558ddff7f7-zh2h7\" (UID: \"6a2a3589-419e-47ba-bc4a-bbc4ddc9ff3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-de97ae-558ddff7f7-zh2h7"
Sep 09 16:30:53.130133 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:53.130012    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a2a3589-419e-47ba-bc4a-bbc4ddc9ff3e-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-de97ae-558ddff7f7-zh2h7\" (UID: \"6a2a3589-419e-47ba-bc4a-bbc4ddc9ff3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-de97ae-558ddff7f7-zh2h7"
Sep 09 16:30:53.230915 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:53.230874    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgprf\" (UniqueName: \"kubernetes.io/projected/6a2a3589-419e-47ba-bc4a-bbc4ddc9ff3e-kube-api-access-xgprf\") pod \"maas-api-e2e-shared-b-w2-de97ae-558ddff7f7-zh2h7\" (UID: \"6a2a3589-419e-47ba-bc4a-bbc4ddc9ff3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-de97ae-558ddff7f7-zh2h7"
Sep 09 16:30:53.230915 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:53.230918    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6a2a3589-419e-47ba-bc4a-bbc4ddc9ff3e-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-de97ae-558ddff7f7-zh2h7\" (UID: \"6a2a3589-419e-47ba-bc4a-bbc4ddc9ff3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-de97ae-558ddff7f7-zh2h7"
Sep 09 16:30:53.231157 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:53.230948    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6a2a3589-419e-47ba-bc4a-bbc4ddc9ff3e-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-de97ae-558ddff7f7-zh2h7\" (UID: \"6a2a3589-419e-47ba-bc4a-bbc4ddc9ff3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-de97ae-558ddff7f7-zh2h7"
Sep 09 16:30:53.231157 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:53.231000    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a2a3589-419e-47ba-bc4a-bbc4ddc9ff3e-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-de97ae-558ddff7f7-zh2h7\" (UID: \"6a2a3589-419e-47ba-bc4a-bbc4ddc9ff3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-de97ae-558ddff7f7-zh2h7"
Sep 09 16:30:53.231853 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:53.231779    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a2a3589-419e-47ba-bc4a-bbc4ddc9ff3e-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-de97ae-558ddff7f7-zh2h7\" (UID: \"6a2a3589-419e-47ba-bc4a-bbc4ddc9ff3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-de97ae-558ddff7f7-zh2h7"
Sep 09 16:30:53.233595 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:53.233574    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6a2a3589-419e-47ba-bc4a-bbc4ddc9ff3e-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-de97ae-558ddff7f7-zh2h7\" (UID: \"6a2a3589-419e-47ba-bc4a-bbc4ddc9ff3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-de97ae-558ddff7f7-zh2h7"
Sep 09 16:30:53.233695 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:53.233684    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6a2a3589-419e-47ba-bc4a-bbc4ddc9ff3e-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-de97ae-558ddff7f7-zh2h7\" (UID: \"6a2a3589-419e-47ba-bc4a-bbc4ddc9ff3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-de97ae-558ddff7f7-zh2h7"
Sep 09 16:30:53.239164 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:53.239143    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgprf\" (UniqueName: \"kubernetes.io/projected/6a2a3589-419e-47ba-bc4a-bbc4ddc9ff3e-kube-api-access-xgprf\") pod \"maas-api-e2e-shared-b-w2-de97ae-558ddff7f7-zh2h7\" (UID: \"6a2a3589-419e-47ba-bc4a-bbc4ddc9ff3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-de97ae-558ddff7f7-zh2h7"
Sep 09 16:30:53.363076 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:53.363042    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-de97ae-558ddff7f7-zh2h7"
Sep 09 16:30:53.640261 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:53.640236    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-de97ae-558ddff7f7-zh2h7"]
Sep 09 16:30:53.642399 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:30:53.642367    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a2a3589_419e_47ba_bc4a_bbc4ddc9ff3e.slice/crio-4405dfcfc9de9ed9d151da7eff3f6dbf7a9126c4e690b0f8e7738fa78527ddbd WatchSource:0}: Error finding container 4405dfcfc9de9ed9d151da7eff3f6dbf7a9126c4e690b0f8e7738fa78527ddbd: Status 404 returned error can't find the container with id 4405dfcfc9de9ed9d151da7eff3f6dbf7a9126c4e690b0f8e7738fa78527ddbd
Sep 09 16:30:53.643879 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:53.643859    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 16:30:53.967570 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:53.967482    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-de97ae-558ddff7f7-zh2h7" event={"ID":"6a2a3589-419e-47ba-bc4a-bbc4ddc9ff3e","Type":"ContainerStarted","Data":"4405dfcfc9de9ed9d151da7eff3f6dbf7a9126c4e690b0f8e7738fa78527ddbd"}
Sep 09 16:30:56.985604 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:56.985568    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-de97ae-558ddff7f7-zh2h7" event={"ID":"6a2a3589-419e-47ba-bc4a-bbc4ddc9ff3e","Type":"ContainerStarted","Data":"8f744da3711135bea583f4d367e067fef993b15b8dd5fcc3a50a2f7383568303"}
Sep 09 16:30:56.986017 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:56.985713    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-de97ae-558ddff7f7-zh2h7"
Sep 09 16:30:57.002511 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:30:57.002433    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-de97ae-558ddff7f7-zh2h7" podStartSLOduration=1.082278811 podStartE2EDuration="4.002419283s" totalImagesPullingTime="2.920140472s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 16:30:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 16:30:57.00131615 +0000 UTC m=+1332.419166180" watchObservedRunningTime="2026-09-09 16:30:57.002419283 +0000 UTC m=+1332.420269281"
Sep 09 16:31:03.003990 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:03.003946    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-de97ae-558ddff7f7-zh2h7"
Sep 09 16:31:04.608749 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:04.608695    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:31:10.064796 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:10.064761    2567 generic.go:356] "Generic (PLEG): container finished" podID="5c55fb39-101d-4ba1-9f9d-ff6a97232826" containerID="1948df54257d60064fa43c7205e1afaa325b400477ed0c89a204244d06ed37c1" exitCode=0
Sep 09 16:31:10.065220 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:10.064836    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m" event={"ID":"5c55fb39-101d-4ba1-9f9d-ff6a97232826","Type":"ContainerDied","Data":"1948df54257d60064fa43c7205e1afaa325b400477ed0c89a204244d06ed37c1"}
Sep 09 16:31:10.065298 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:10.065284    2567 scope.go:117] "RemoveContainer" containerID="1948df54257d60064fa43c7205e1afaa325b400477ed0c89a204244d06ed37c1"
Sep 09 16:31:10.066405 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:10.066376    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:31:10.066477 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:10.066449    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:31:10.066528 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:10.066510    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:31:11.072299 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:11.072267    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m" event={"ID":"5c55fb39-101d-4ba1-9f9d-ff6a97232826","Type":"ContainerStarted","Data":"9aa6ce252c047ad10ff0d02759c68d2bfaac84ae6ab277970644f5ffe633374c"}
Sep 09 16:31:11.676725 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:11.676689    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m"
Sep 09 16:31:11.676907 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:11.676851    2567 patch_prober.go:28] interesting pod/e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Sep 09 16:31:11.676951 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:11.676899    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m" podUID="5c55fb39-101d-4ba1-9f9d-ff6a97232826" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Sep 09 16:31:12.677049 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:12.677011    2567 patch_prober.go:28] interesting pod/e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Sep 09 16:31:12.677545 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:12.677074    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m" podUID="5c55fb39-101d-4ba1-9f9d-ff6a97232826" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Sep 09 16:31:13.677385 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:13.677346    2567 patch_prober.go:28] interesting pod/e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Sep 09 16:31:13.677826 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:13.677412    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m" podUID="5c55fb39-101d-4ba1-9f9d-ff6a97232826" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Sep 09 16:31:14.677729 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:14.677692    2567 patch_prober.go:28] interesting pod/e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Sep 09 16:31:14.678133 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:14.677754    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m" podUID="5c55fb39-101d-4ba1-9f9d-ff6a97232826" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Sep 09 16:31:15.676782 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:15.676727    2567 patch_prober.go:28] interesting pod/e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Sep 09 16:31:15.676963 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:15.676797    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m" podUID="5c55fb39-101d-4ba1-9f9d-ff6a97232826" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Sep 09 16:31:16.681644 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:16.681612    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m"
Sep 09 16:31:16.682080 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:16.681902    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m"
Sep 09 16:31:16.682544 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:16.682527    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m"
Sep 09 16:31:25.814288 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:25.814248    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:31:33.743627 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:33.743591    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87"]
Sep 09 16:31:33.748943 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:33.748920    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87"
Sep 09 16:31:33.751530 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:33.751442    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-a0d4fa04-openshift-default-dockercfg-z95sg\""
Sep 09 16:31:33.759821 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:33.759799    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87"]
Sep 09 16:31:33.835493 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:33.833988    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae0afbee-c912-40f1-8e93-3db00cde2595-workload-socket\") pod \"e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87\" (UID: \"ae0afbee-c912-40f1-8e93-3db00cde2595\") " pod="openshift-ingress/e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87"
Sep 09 16:31:33.835493 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:33.834038    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae0afbee-c912-40f1-8e93-3db00cde2595-workload-certs\") pod \"e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87\" (UID: \"ae0afbee-c912-40f1-8e93-3db00cde2595\") " pod="openshift-ingress/e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87"
Sep 09 16:31:33.835493 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:33.834087    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae0afbee-c912-40f1-8e93-3db00cde2595-istiod-ca-cert\") pod \"e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87\" (UID: \"ae0afbee-c912-40f1-8e93-3db00cde2595\") " pod="openshift-ingress/e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87"
Sep 09 16:31:33.835493 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:33.834122    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwngz\" (UniqueName: \"kubernetes.io/projected/ae0afbee-c912-40f1-8e93-3db00cde2595-kube-api-access-rwngz\") pod \"e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87\" (UID: \"ae0afbee-c912-40f1-8e93-3db00cde2595\") " pod="openshift-ingress/e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87"
Sep 09 16:31:33.835493 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:33.834164    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae0afbee-c912-40f1-8e93-3db00cde2595-credential-socket\") pod \"e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87\" (UID: \"ae0afbee-c912-40f1-8e93-3db00cde2595\") " pod="openshift-ingress/e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87"
Sep 09 16:31:33.835493 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:33.834189    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae0afbee-c912-40f1-8e93-3db00cde2595-istio-token\") pod \"e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87\" (UID: \"ae0afbee-c912-40f1-8e93-3db00cde2595\") " pod="openshift-ingress/e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87"
Sep 09 16:31:33.835493 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:33.834249    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae0afbee-c912-40f1-8e93-3db00cde2595-istio-podinfo\") pod \"e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87\" (UID: \"ae0afbee-c912-40f1-8e93-3db00cde2595\") " pod="openshift-ingress/e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87"
Sep 09 16:31:33.835493 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:33.834292    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae0afbee-c912-40f1-8e93-3db00cde2595-istio-data\") pod \"e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87\" (UID: \"ae0afbee-c912-40f1-8e93-3db00cde2595\") " pod="openshift-ingress/e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87"
Sep 09 16:31:33.835493 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:33.834317    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae0afbee-c912-40f1-8e93-3db00cde2595-istio-envoy\") pod \"e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87\" (UID: \"ae0afbee-c912-40f1-8e93-3db00cde2595\") " pod="openshift-ingress/e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87"
Sep 09 16:31:33.935713 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:33.935675    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae0afbee-c912-40f1-8e93-3db00cde2595-istio-data\") pod \"e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87\" (UID: \"ae0afbee-c912-40f1-8e93-3db00cde2595\") " pod="openshift-ingress/e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87"
Sep 09 16:31:33.935713 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:33.935716    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae0afbee-c912-40f1-8e93-3db00cde2595-istio-envoy\") pod \"e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87\" (UID: \"ae0afbee-c912-40f1-8e93-3db00cde2595\") " pod="openshift-ingress/e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87"
Sep 09 16:31:33.935988 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:33.935793    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae0afbee-c912-40f1-8e93-3db00cde2595-workload-socket\") pod \"e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87\" (UID: \"ae0afbee-c912-40f1-8e93-3db00cde2595\") " pod="openshift-ingress/e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87"
Sep 09 16:31:33.935988 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:33.935808    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae0afbee-c912-40f1-8e93-3db00cde2595-workload-certs\") pod \"e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87\" (UID: \"ae0afbee-c912-40f1-8e93-3db00cde2595\") " pod="openshift-ingress/e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87"
Sep 09 16:31:33.935988 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:33.935838    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae0afbee-c912-40f1-8e93-3db00cde2595-istiod-ca-cert\") pod \"e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87\" (UID: \"ae0afbee-c912-40f1-8e93-3db00cde2595\") " pod="openshift-ingress/e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87"
Sep 09 16:31:33.935988 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:33.935864    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwngz\" (UniqueName: \"kubernetes.io/projected/ae0afbee-c912-40f1-8e93-3db00cde2595-kube-api-access-rwngz\") pod \"e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87\" (UID: \"ae0afbee-c912-40f1-8e93-3db00cde2595\") " pod="openshift-ingress/e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87"
Sep 09 16:31:33.935988 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:33.935896    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae0afbee-c912-40f1-8e93-3db00cde2595-credential-socket\") pod \"e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87\" (UID: \"ae0afbee-c912-40f1-8e93-3db00cde2595\") " pod="openshift-ingress/e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87"
Sep 09 16:31:33.935988 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:33.935920    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae0afbee-c912-40f1-8e93-3db00cde2595-istio-token\") pod \"e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87\" (UID: \"ae0afbee-c912-40f1-8e93-3db00cde2595\") " pod="openshift-ingress/e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87"
Sep 09 16:31:33.935988 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:33.935966    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae0afbee-c912-40f1-8e93-3db00cde2595-istio-podinfo\") pod \"e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87\" (UID: \"ae0afbee-c912-40f1-8e93-3db00cde2595\") " pod="openshift-ingress/e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87"
Sep 09 16:31:33.936369 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:33.936156    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae0afbee-c912-40f1-8e93-3db00cde2595-istio-data\") pod \"e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87\" (UID: \"ae0afbee-c912-40f1-8e93-3db00cde2595\") " pod="openshift-ingress/e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87"
Sep 09 16:31:33.936369 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:33.936231    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae0afbee-c912-40f1-8e93-3db00cde2595-workload-socket\") pod \"e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87\" (UID: \"ae0afbee-c912-40f1-8e93-3db00cde2595\") " pod="openshift-ingress/e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87"
Sep 09 16:31:33.936369 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:33.936271    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae0afbee-c912-40f1-8e93-3db00cde2595-workload-certs\") pod \"e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87\" (UID: \"ae0afbee-c912-40f1-8e93-3db00cde2595\") " pod="openshift-ingress/e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87"
Sep 09 16:31:33.936731 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:33.936494    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae0afbee-c912-40f1-8e93-3db00cde2595-credential-socket\") pod \"e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87\" (UID: \"ae0afbee-c912-40f1-8e93-3db00cde2595\") " pod="openshift-ingress/e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87"
Sep 09 16:31:33.936885 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:33.936852    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae0afbee-c912-40f1-8e93-3db00cde2595-istiod-ca-cert\") pod \"e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87\" (UID: \"ae0afbee-c912-40f1-8e93-3db00cde2595\") " pod="openshift-ingress/e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87"
Sep 09 16:31:33.938263 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:33.938241    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae0afbee-c912-40f1-8e93-3db00cde2595-istio-envoy\") pod \"e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87\" (UID: \"ae0afbee-c912-40f1-8e93-3db00cde2595\") " pod="openshift-ingress/e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87"
Sep 09 16:31:33.938596 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:33.938578    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae0afbee-c912-40f1-8e93-3db00cde2595-istio-podinfo\") pod \"e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87\" (UID: \"ae0afbee-c912-40f1-8e93-3db00cde2595\") " pod="openshift-ingress/e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87"
Sep 09 16:31:33.943523 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:33.943504    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae0afbee-c912-40f1-8e93-3db00cde2595-istio-token\") pod \"e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87\" (UID: \"ae0afbee-c912-40f1-8e93-3db00cde2595\") " pod="openshift-ingress/e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87"
Sep 09 16:31:33.943702 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:33.943686    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwngz\" (UniqueName: \"kubernetes.io/projected/ae0afbee-c912-40f1-8e93-3db00cde2595-kube-api-access-rwngz\") pod \"e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87\" (UID: \"ae0afbee-c912-40f1-8e93-3db00cde2595\") " pod="openshift-ingress/e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87"
Sep 09 16:31:34.060484 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:34.060365    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87"
Sep 09 16:31:34.338322 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:34.338296    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87"]
Sep 09 16:31:34.340448 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:31:34.340413    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae0afbee_c912_40f1_8e93_3db00cde2595.slice/crio-d9527f6ff1b5f419503766054af2288ac0df04f921b1ba83904ed134059bf1aa WatchSource:0}: Error finding container d9527f6ff1b5f419503766054af2288ac0df04f921b1ba83904ed134059bf1aa: Status 404 returned error can't find the container with id d9527f6ff1b5f419503766054af2288ac0df04f921b1ba83904ed134059bf1aa
Sep 09 16:31:34.342772 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:34.342736    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:31:34.342888 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:34.342825    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:31:34.342888 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:34.342870    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:31:35.202514 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:35.202473    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87" event={"ID":"ae0afbee-c912-40f1-8e93-3db00cde2595","Type":"ContainerStarted","Data":"dde86748e8687e1a528519c4f479038794040af556a65125f5aa5d359a816686"}
Sep 09 16:31:35.202922 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:35.202537    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87" event={"ID":"ae0afbee-c912-40f1-8e93-3db00cde2595","Type":"ContainerStarted","Data":"d9527f6ff1b5f419503766054af2288ac0df04f921b1ba83904ed134059bf1aa"}
Sep 09 16:31:35.222725 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:35.222672    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87" podStartSLOduration=2.222656491 podStartE2EDuration="2.222656491s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 16:31:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 16:31:35.220305438 +0000 UTC m=+1370.638155449" watchObservedRunningTime="2026-09-09 16:31:35.222656491 +0000 UTC m=+1370.640506537"
Sep 09 16:31:36.060765 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:36.060719    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87"
Sep 09 16:31:36.066046 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:36.066017    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87"
Sep 09 16:31:36.207773 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:36.207739    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87"
Sep 09 16:31:36.208567 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:36.208546    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87"
Sep 09 16:31:42.919422 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:42.919384    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:31:45.123793 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:45.123762    2567 scope.go:117] "RemoveContainer" containerID="5a7f3da675cc8ea7ec5087b778301c19f38462c0b6054f9e0b62b1c8be0246ff"
Sep 09 16:31:52.249653 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:52.249609    2567 patch_prober.go:28] interesting pod/e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 09 16:31:52.250056 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:52.249667    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87" podUID="ae0afbee-c912-40f1-8e93-3db00cde2595" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 09 16:31:53.297835 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:53.297740    2567 generic.go:356] "Generic (PLEG): container finished" podID="ae0afbee-c912-40f1-8e93-3db00cde2595" containerID="dde86748e8687e1a528519c4f479038794040af556a65125f5aa5d359a816686" exitCode=0
Sep 09 16:31:53.297835 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:53.297815    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87" event={"ID":"ae0afbee-c912-40f1-8e93-3db00cde2595","Type":"ContainerDied","Data":"dde86748e8687e1a528519c4f479038794040af556a65125f5aa5d359a816686"}
Sep 09 16:31:53.298258 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:53.298234    2567 scope.go:117] "RemoveContainer" containerID="dde86748e8687e1a528519c4f479038794040af556a65125f5aa5d359a816686"
Sep 09 16:31:53.299414 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:53.299385    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:31:53.299485 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:53.299474    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:31:53.299530 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:53.299511    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:31:54.313956 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:54.313921    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87" event={"ID":"ae0afbee-c912-40f1-8e93-3db00cde2595","Type":"ContainerStarted","Data":"b5f2a60814264b114a3d136ccc4ece17305f504fbc59a3c67e8a0ad1cbf7e161"}
Sep 09 16:31:55.061449 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:55.061411    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87"
Sep 09 16:31:55.061702 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:55.061660    2567 patch_prober.go:28] interesting pod/e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 09 16:31:55.061786 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:55.061735    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87" podUID="ae0afbee-c912-40f1-8e93-3db00cde2595" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 09 16:31:56.061304 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:56.061264    2567 patch_prober.go:28] interesting pod/e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 09 16:31:56.061742 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:56.061321    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87" podUID="ae0afbee-c912-40f1-8e93-3db00cde2595" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 09 16:31:57.061327 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:57.061288    2567 patch_prober.go:28] interesting pod/e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 09 16:31:57.061759 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:57.061345    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87" podUID="ae0afbee-c912-40f1-8e93-3db00cde2595" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 09 16:31:58.061214 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:58.061175    2567 patch_prober.go:28] interesting pod/e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 09 16:31:58.061400 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:58.061236    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87" podUID="ae0afbee-c912-40f1-8e93-3db00cde2595" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 09 16:31:59.060899 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:59.060862    2567 patch_prober.go:28] interesting pod/e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 09 16:31:59.061080 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:31:59.060919    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87" podUID="ae0afbee-c912-40f1-8e93-3db00cde2595" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 09 16:32:00.064353 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:32:00.064317    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87"
Sep 09 16:32:00.064801 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:32:00.064667    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87"
Sep 09 16:32:00.065271 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:32:00.065254    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87"
Sep 09 16:32:00.608010 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:32:00.607974    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:32:16.218105 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:32:16.218068    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:32:34.214234 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:32:34.214189    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:32:45.143539 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:32:45.143439    2567 scope.go:117] "RemoveContainer" containerID="ded8c57af835b9b33dd8d85e91593d2d6e1316182c7a7137e62149a85aa0223c"
Sep 09 16:32:51.107556 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:32:51.107518    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:33:12.413410 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:33:12.413372    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:33:29.907474 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:33:29.907429    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:33:32.574760 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:33:32.574717    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87"]
Sep 09 16:33:32.575705 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:33:32.575637    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87" podUID="ae0afbee-c912-40f1-8e93-3db00cde2595" containerName="istio-proxy" containerID="cri-o://b5f2a60814264b114a3d136ccc4ece17305f504fbc59a3c67e8a0ad1cbf7e161" gracePeriod=30
Sep 09 16:33:37.837694 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:33:37.837670    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87"
Sep 09 16:33:37.878557 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:33:37.878437    2567 generic.go:356] "Generic (PLEG): container finished" podID="ae0afbee-c912-40f1-8e93-3db00cde2595" containerID="b5f2a60814264b114a3d136ccc4ece17305f504fbc59a3c67e8a0ad1cbf7e161" exitCode=0
Sep 09 16:33:37.878557 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:33:37.878516    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87"
Sep 09 16:33:37.878557 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:33:37.878519    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87" event={"ID":"ae0afbee-c912-40f1-8e93-3db00cde2595","Type":"ContainerDied","Data":"b5f2a60814264b114a3d136ccc4ece17305f504fbc59a3c67e8a0ad1cbf7e161"}
Sep 09 16:33:37.878788 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:33:37.878584    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87" event={"ID":"ae0afbee-c912-40f1-8e93-3db00cde2595","Type":"ContainerDied","Data":"d9527f6ff1b5f419503766054af2288ac0df04f921b1ba83904ed134059bf1aa"}
Sep 09 16:33:37.878788 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:33:37.878616    2567 scope.go:117] "RemoveContainer" containerID="b5f2a60814264b114a3d136ccc4ece17305f504fbc59a3c67e8a0ad1cbf7e161"
Sep 09 16:33:37.887852 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:33:37.887832    2567 scope.go:117] "RemoveContainer" containerID="dde86748e8687e1a528519c4f479038794040af556a65125f5aa5d359a816686"
Sep 09 16:33:37.896173 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:33:37.896152    2567 scope.go:117] "RemoveContainer" containerID="b5f2a60814264b114a3d136ccc4ece17305f504fbc59a3c67e8a0ad1cbf7e161"
Sep 09 16:33:37.896472 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:33:37.896442    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5f2a60814264b114a3d136ccc4ece17305f504fbc59a3c67e8a0ad1cbf7e161\": container with ID starting with b5f2a60814264b114a3d136ccc4ece17305f504fbc59a3c67e8a0ad1cbf7e161 not found: ID does not exist" containerID="b5f2a60814264b114a3d136ccc4ece17305f504fbc59a3c67e8a0ad1cbf7e161"
Sep 09 16:33:37.896538 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:33:37.896484    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5f2a60814264b114a3d136ccc4ece17305f504fbc59a3c67e8a0ad1cbf7e161"} err="failed to get container status \"b5f2a60814264b114a3d136ccc4ece17305f504fbc59a3c67e8a0ad1cbf7e161\": rpc error: code = NotFound desc = could not find container \"b5f2a60814264b114a3d136ccc4ece17305f504fbc59a3c67e8a0ad1cbf7e161\": container with ID starting with b5f2a60814264b114a3d136ccc4ece17305f504fbc59a3c67e8a0ad1cbf7e161 not found: ID does not exist"
Sep 09 16:33:37.896538 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:33:37.896502    2567 scope.go:117] "RemoveContainer" containerID="dde86748e8687e1a528519c4f479038794040af556a65125f5aa5d359a816686"
Sep 09 16:33:37.896756 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:33:37.896724    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dde86748e8687e1a528519c4f479038794040af556a65125f5aa5d359a816686\": container with ID starting with dde86748e8687e1a528519c4f479038794040af556a65125f5aa5d359a816686 not found: ID does not exist" containerID="dde86748e8687e1a528519c4f479038794040af556a65125f5aa5d359a816686"
Sep 09 16:33:37.896800 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:33:37.896760    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dde86748e8687e1a528519c4f479038794040af556a65125f5aa5d359a816686"} err="failed to get container status \"dde86748e8687e1a528519c4f479038794040af556a65125f5aa5d359a816686\": rpc error: code = NotFound desc = could not find container \"dde86748e8687e1a528519c4f479038794040af556a65125f5aa5d359a816686\": container with ID starting with dde86748e8687e1a528519c4f479038794040af556a65125f5aa5d359a816686 not found: ID does not exist"
Sep 09 16:33:37.901099 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:33:37.901080    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae0afbee-c912-40f1-8e93-3db00cde2595-istio-podinfo\") pod \"ae0afbee-c912-40f1-8e93-3db00cde2595\" (UID: \"ae0afbee-c912-40f1-8e93-3db00cde2595\") "
Sep 09 16:33:37.901174 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:33:37.901124    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae0afbee-c912-40f1-8e93-3db00cde2595-workload-socket\") pod \"ae0afbee-c912-40f1-8e93-3db00cde2595\" (UID: \"ae0afbee-c912-40f1-8e93-3db00cde2595\") "
Sep 09 16:33:37.901174 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:33:37.901159    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae0afbee-c912-40f1-8e93-3db00cde2595-workload-certs\") pod \"ae0afbee-c912-40f1-8e93-3db00cde2595\" (UID: \"ae0afbee-c912-40f1-8e93-3db00cde2595\") "
Sep 09 16:33:37.901268 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:33:37.901252    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae0afbee-c912-40f1-8e93-3db00cde2595-istio-data\") pod \"ae0afbee-c912-40f1-8e93-3db00cde2595\" (UID: \"ae0afbee-c912-40f1-8e93-3db00cde2595\") "
Sep 09 16:33:37.901312 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:33:37.901293    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae0afbee-c912-40f1-8e93-3db00cde2595-istiod-ca-cert\") pod \"ae0afbee-c912-40f1-8e93-3db00cde2595\" (UID: \"ae0afbee-c912-40f1-8e93-3db00cde2595\") "
Sep 09 16:33:37.901351 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:33:37.901333    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae0afbee-c912-40f1-8e93-3db00cde2595-istio-token\") pod \"ae0afbee-c912-40f1-8e93-3db00cde2595\" (UID: \"ae0afbee-c912-40f1-8e93-3db00cde2595\") "
Sep 09 16:33:37.901392 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:33:37.901367    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae0afbee-c912-40f1-8e93-3db00cde2595-istio-envoy\") pod \"ae0afbee-c912-40f1-8e93-3db00cde2595\" (UID: \"ae0afbee-c912-40f1-8e93-3db00cde2595\") "
Sep 09 16:33:37.901392 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:33:37.901382    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae0afbee-c912-40f1-8e93-3db00cde2595-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ae0afbee-c912-40f1-8e93-3db00cde2595" (UID: "ae0afbee-c912-40f1-8e93-3db00cde2595"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:33:37.901518 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:33:37.901410    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae0afbee-c912-40f1-8e93-3db00cde2595-credential-socket\") pod \"ae0afbee-c912-40f1-8e93-3db00cde2595\" (UID: \"ae0afbee-c912-40f1-8e93-3db00cde2595\") "
Sep 09 16:33:37.901518 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:33:37.901422    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae0afbee-c912-40f1-8e93-3db00cde2595-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ae0afbee-c912-40f1-8e93-3db00cde2595" (UID: "ae0afbee-c912-40f1-8e93-3db00cde2595"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:33:37.901518 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:33:37.901482    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rwngz\" (UniqueName: \"kubernetes.io/projected/ae0afbee-c912-40f1-8e93-3db00cde2595-kube-api-access-rwngz\") pod \"ae0afbee-c912-40f1-8e93-3db00cde2595\" (UID: \"ae0afbee-c912-40f1-8e93-3db00cde2595\") "
Sep 09 16:33:37.901518 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:33:37.901515    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae0afbee-c912-40f1-8e93-3db00cde2595-istio-data" (OuterVolumeSpecName: "istio-data") pod "ae0afbee-c912-40f1-8e93-3db00cde2595" (UID: "ae0afbee-c912-40f1-8e93-3db00cde2595"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:33:37.901728 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:33:37.901677    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae0afbee-c912-40f1-8e93-3db00cde2595-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ae0afbee-c912-40f1-8e93-3db00cde2595" (UID: "ae0afbee-c912-40f1-8e93-3db00cde2595"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:33:37.901789 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:33:37.901719    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae0afbee-c912-40f1-8e93-3db00cde2595-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ae0afbee-c912-40f1-8e93-3db00cde2595" (UID: "ae0afbee-c912-40f1-8e93-3db00cde2595"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 16:33:37.901939 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:33:37.901920    2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae0afbee-c912-40f1-8e93-3db00cde2595-workload-socket\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:33:37.902163 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:33:37.901946    2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae0afbee-c912-40f1-8e93-3db00cde2595-workload-certs\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:33:37.902163 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:33:37.901961    2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae0afbee-c912-40f1-8e93-3db00cde2595-istio-data\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:33:37.902163 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:33:37.901974    2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae0afbee-c912-40f1-8e93-3db00cde2595-istiod-ca-cert\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:33:37.902163 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:33:37.901988    2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae0afbee-c912-40f1-8e93-3db00cde2595-credential-socket\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:33:37.903671 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:33:37.903640    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ae0afbee-c912-40f1-8e93-3db00cde2595-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ae0afbee-c912-40f1-8e93-3db00cde2595" (UID: "ae0afbee-c912-40f1-8e93-3db00cde2595"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 16:33:37.903764 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:33:37.903668    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae0afbee-c912-40f1-8e93-3db00cde2595-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ae0afbee-c912-40f1-8e93-3db00cde2595" (UID: "ae0afbee-c912-40f1-8e93-3db00cde2595"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:33:37.903764 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:33:37.903719    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae0afbee-c912-40f1-8e93-3db00cde2595-istio-token" (OuterVolumeSpecName: "istio-token") pod "ae0afbee-c912-40f1-8e93-3db00cde2595" (UID: "ae0afbee-c912-40f1-8e93-3db00cde2595"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 16:33:37.903764 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:33:37.903741    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae0afbee-c912-40f1-8e93-3db00cde2595-kube-api-access-rwngz" (OuterVolumeSpecName: "kube-api-access-rwngz") pod "ae0afbee-c912-40f1-8e93-3db00cde2595" (UID: "ae0afbee-c912-40f1-8e93-3db00cde2595"). InnerVolumeSpecName "kube-api-access-rwngz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 16:33:38.003297 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:33:38.003262    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rwngz\" (UniqueName: \"kubernetes.io/projected/ae0afbee-c912-40f1-8e93-3db00cde2595-kube-api-access-rwngz\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:33:38.003297 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:33:38.003288    2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae0afbee-c912-40f1-8e93-3db00cde2595-istio-podinfo\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:33:38.003297 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:33:38.003298    2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae0afbee-c912-40f1-8e93-3db00cde2595-istio-token\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:33:38.003297 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:33:38.003305    2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae0afbee-c912-40f1-8e93-3db00cde2595-istio-envoy\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:33:38.200359 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:33:38.200329    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87"]
Sep 09 16:33:38.203359 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:33:38.203334    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-a0d4fa04-openshift-default-544b6595fb-vhb87"]
Sep 09 16:33:39.036051 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:33:39.036018    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae0afbee-c912-40f1-8e93-3db00cde2595" path="/var/lib/kubelet/pods/ae0afbee-c912-40f1-8e93-3db00cde2595/volumes"
Sep 09 16:33:45.143576 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:33:45.143540    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-xwlzs_16ecfb4d-7944-465e-a38e-d8c5bda6b1ac/console-operator/2.log"
Sep 09 16:33:45.152405 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:33:45.152379    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-xwlzs_16ecfb4d-7944-465e-a38e-d8c5bda6b1ac/console-operator/2.log"
Sep 09 16:33:45.166752 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:33:45.166723    2567 scope.go:117] "RemoveContainer" containerID="ec6a444c756149d0e61b70a04538b0bc284b9d9d39da5c6112b8ec3e2bfcf625"
Sep 09 16:33:45.175013 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:33:45.174993    2567 scope.go:117] "RemoveContainer" containerID="876c837b7708cc91ec440fbfb9a2eac37df9081fd1202d83164589bae3a92fef"
Sep 09 16:33:45.182903 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:33:45.182884    2567 scope.go:117] "RemoveContainer" containerID="b310bced0a26fd152d3cbbfb2ba1dc46540aaaa24bd6e31b3ac497cc9eef51de"
Sep 09 16:33:47.907798 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:33:47.907761    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:34:06.711072 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:06.710985    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:34:27.772081 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:27.772041    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9"]
Sep 09 16:34:27.772651 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:27.772633    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae0afbee-c912-40f1-8e93-3db00cde2595" containerName="istio-proxy"
Sep 09 16:34:27.772724 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:27.772653    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae0afbee-c912-40f1-8e93-3db00cde2595" containerName="istio-proxy"
Sep 09 16:34:27.772783 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:27.772763    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae0afbee-c912-40f1-8e93-3db00cde2595" containerName="istio-proxy"
Sep 09 16:34:27.772783 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:27.772773    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae0afbee-c912-40f1-8e93-3db00cde2595" containerName="istio-proxy"
Sep 09 16:34:27.776348 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:27.776325    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9"
Sep 09 16:34:27.778658 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:27.778635    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-5d8327-openshift-default-dockercfg-jhcng\""
Sep 09 16:34:27.788611 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:27.788586    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9"]
Sep 09 16:34:27.886269 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:27.886225    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8dedd14-b232-4636-9f89-8552b16c6d94-istio-data\") pod \"e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9\" (UID: \"e8dedd14-b232-4636-9f89-8552b16c6d94\") " pod="openshift-ingress/e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9"
Sep 09 16:34:27.886603 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:27.886575    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8dedd14-b232-4636-9f89-8552b16c6d94-workload-socket\") pod \"e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9\" (UID: \"e8dedd14-b232-4636-9f89-8552b16c6d94\") " pod="openshift-ingress/e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9"
Sep 09 16:34:27.886760 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:27.886744    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8dedd14-b232-4636-9f89-8552b16c6d94-istio-token\") pod \"e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9\" (UID: \"e8dedd14-b232-4636-9f89-8552b16c6d94\") " pod="openshift-ingress/e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9"
Sep 09 16:34:27.886877 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:27.886862    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8dedd14-b232-4636-9f89-8552b16c6d94-istio-envoy\") pod \"e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9\" (UID: \"e8dedd14-b232-4636-9f89-8552b16c6d94\") " pod="openshift-ingress/e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9"
Sep 09 16:34:27.887056 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:27.887037    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8dedd14-b232-4636-9f89-8552b16c6d94-credential-socket\") pod \"e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9\" (UID: \"e8dedd14-b232-4636-9f89-8552b16c6d94\") " pod="openshift-ingress/e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9"
Sep 09 16:34:27.887167 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:27.887151    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8dedd14-b232-4636-9f89-8552b16c6d94-istio-podinfo\") pod \"e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9\" (UID: \"e8dedd14-b232-4636-9f89-8552b16c6d94\") " pod="openshift-ingress/e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9"
Sep 09 16:34:27.887310 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:27.887293    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8dedd14-b232-4636-9f89-8552b16c6d94-workload-certs\") pod \"e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9\" (UID: \"e8dedd14-b232-4636-9f89-8552b16c6d94\") " pod="openshift-ingress/e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9"
Sep 09 16:34:27.887500 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:27.887481    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8dedd14-b232-4636-9f89-8552b16c6d94-istiod-ca-cert\") pod \"e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9\" (UID: \"e8dedd14-b232-4636-9f89-8552b16c6d94\") " pod="openshift-ingress/e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9"
Sep 09 16:34:27.887732 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:27.887717    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jh9jg\" (UniqueName: \"kubernetes.io/projected/e8dedd14-b232-4636-9f89-8552b16c6d94-kube-api-access-jh9jg\") pod \"e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9\" (UID: \"e8dedd14-b232-4636-9f89-8552b16c6d94\") " pod="openshift-ingress/e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9"
Sep 09 16:34:27.988654 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:27.988622    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8dedd14-b232-4636-9f89-8552b16c6d94-workload-certs\") pod \"e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9\" (UID: \"e8dedd14-b232-4636-9f89-8552b16c6d94\") " pod="openshift-ingress/e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9"
Sep 09 16:34:27.988827 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:27.988672    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8dedd14-b232-4636-9f89-8552b16c6d94-istiod-ca-cert\") pod \"e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9\" (UID: \"e8dedd14-b232-4636-9f89-8552b16c6d94\") " pod="openshift-ingress/e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9"
Sep 09 16:34:27.988827 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:27.988718    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jh9jg\" (UniqueName: \"kubernetes.io/projected/e8dedd14-b232-4636-9f89-8552b16c6d94-kube-api-access-jh9jg\") pod \"e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9\" (UID: \"e8dedd14-b232-4636-9f89-8552b16c6d94\") " pod="openshift-ingress/e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9"
Sep 09 16:34:27.988827 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:27.988738    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8dedd14-b232-4636-9f89-8552b16c6d94-istio-data\") pod \"e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9\" (UID: \"e8dedd14-b232-4636-9f89-8552b16c6d94\") " pod="openshift-ingress/e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9"
Sep 09 16:34:27.988827 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:27.988760    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8dedd14-b232-4636-9f89-8552b16c6d94-workload-socket\") pod \"e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9\" (UID: \"e8dedd14-b232-4636-9f89-8552b16c6d94\") " pod="openshift-ingress/e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9"
Sep 09 16:34:27.989037 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:27.988870    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8dedd14-b232-4636-9f89-8552b16c6d94-istio-token\") pod \"e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9\" (UID: \"e8dedd14-b232-4636-9f89-8552b16c6d94\") " pod="openshift-ingress/e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9"
Sep 09 16:34:27.989037 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:27.988904    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8dedd14-b232-4636-9f89-8552b16c6d94-istio-envoy\") pod \"e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9\" (UID: \"e8dedd14-b232-4636-9f89-8552b16c6d94\") " pod="openshift-ingress/e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9"
Sep 09 16:34:27.989037 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:27.988987    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8dedd14-b232-4636-9f89-8552b16c6d94-credential-socket\") pod \"e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9\" (UID: \"e8dedd14-b232-4636-9f89-8552b16c6d94\") " pod="openshift-ingress/e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9"
Sep 09 16:34:27.989037 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:27.989013    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8dedd14-b232-4636-9f89-8552b16c6d94-istio-podinfo\") pod \"e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9\" (UID: \"e8dedd14-b232-4636-9f89-8552b16c6d94\") " pod="openshift-ingress/e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9"
Sep 09 16:34:27.989247 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:27.989071    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8dedd14-b232-4636-9f89-8552b16c6d94-workload-certs\") pod \"e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9\" (UID: \"e8dedd14-b232-4636-9f89-8552b16c6d94\") " pod="openshift-ingress/e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9"
Sep 09 16:34:27.989247 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:27.989122    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8dedd14-b232-4636-9f89-8552b16c6d94-istio-data\") pod \"e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9\" (UID: \"e8dedd14-b232-4636-9f89-8552b16c6d94\") " pod="openshift-ingress/e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9"
Sep 09 16:34:27.989481 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:27.989444    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8dedd14-b232-4636-9f89-8552b16c6d94-workload-socket\") pod \"e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9\" (UID: \"e8dedd14-b232-4636-9f89-8552b16c6d94\") " pod="openshift-ingress/e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9"
Sep 09 16:34:27.989561 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:27.989478    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8dedd14-b232-4636-9f89-8552b16c6d94-istiod-ca-cert\") pod \"e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9\" (UID: \"e8dedd14-b232-4636-9f89-8552b16c6d94\") " pod="openshift-ingress/e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9"
Sep 09 16:34:27.989600 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:27.989587    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8dedd14-b232-4636-9f89-8552b16c6d94-credential-socket\") pod \"e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9\" (UID: \"e8dedd14-b232-4636-9f89-8552b16c6d94\") " pod="openshift-ingress/e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9"
Sep 09 16:34:27.991412 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:27.991386    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8dedd14-b232-4636-9f89-8552b16c6d94-istio-envoy\") pod \"e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9\" (UID: \"e8dedd14-b232-4636-9f89-8552b16c6d94\") " pod="openshift-ingress/e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9"
Sep 09 16:34:27.991632 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:27.991612    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8dedd14-b232-4636-9f89-8552b16c6d94-istio-podinfo\") pod \"e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9\" (UID: \"e8dedd14-b232-4636-9f89-8552b16c6d94\") " pod="openshift-ingress/e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9"
Sep 09 16:34:27.995997 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:27.995970    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jh9jg\" (UniqueName: \"kubernetes.io/projected/e8dedd14-b232-4636-9f89-8552b16c6d94-kube-api-access-jh9jg\") pod \"e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9\" (UID: \"e8dedd14-b232-4636-9f89-8552b16c6d94\") " pod="openshift-ingress/e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9"
Sep 09 16:34:27.996167 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:27.996148    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8dedd14-b232-4636-9f89-8552b16c6d94-istio-token\") pod \"e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9\" (UID: \"e8dedd14-b232-4636-9f89-8552b16c6d94\") " pod="openshift-ingress/e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9"
Sep 09 16:34:28.088118 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:28.088082    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9"
Sep 09 16:34:28.363100 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:28.363072    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9"]
Sep 09 16:34:28.365172 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:34:28.365149    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8dedd14_b232_4636_9f89_8552b16c6d94.slice/crio-7f31c3a6277e2cc34db6c174a3ffb0b351e56d38252b828e6de003d509be2902 WatchSource:0}: Error finding container 7f31c3a6277e2cc34db6c174a3ffb0b351e56d38252b828e6de003d509be2902: Status 404 returned error can't find the container with id 7f31c3a6277e2cc34db6c174a3ffb0b351e56d38252b828e6de003d509be2902
Sep 09 16:34:28.367578 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:28.367543    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:34:28.367661 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:28.367612    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:34:28.367661 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:28.367639    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:34:29.151716 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:29.151674    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9" event={"ID":"e8dedd14-b232-4636-9f89-8552b16c6d94","Type":"ContainerStarted","Data":"884d2a0324ebb757743bfd0e8dbcab01c8fec602b362586fef48e28e83bc693f"}
Sep 09 16:34:29.152169 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:29.151733    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9" event={"ID":"e8dedd14-b232-4636-9f89-8552b16c6d94","Type":"ContainerStarted","Data":"7f31c3a6277e2cc34db6c174a3ffb0b351e56d38252b828e6de003d509be2902"}
Sep 09 16:34:29.173723 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:29.173668    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9" podStartSLOduration=2.173652877 podStartE2EDuration="2.173652877s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 16:34:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 16:34:29.171443526 +0000 UTC m=+1544.589293553" watchObservedRunningTime="2026-09-09 16:34:29.173652877 +0000 UTC m=+1544.591502875"
Sep 09 16:34:30.089018 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:30.088980    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9"
Sep 09 16:34:30.094795 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:30.094767    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9"
Sep 09 16:34:30.156397 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:30.156369    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9"
Sep 09 16:34:30.157237 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:30.157216    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9"
Sep 09 16:34:32.510997 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:32.510963    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:34:45.214741 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:45.214707    2567 scope.go:117] "RemoveContainer" containerID="06efe5d24781058a08f46b2ac5a940a008d69b3e6acb8518a92b560528e60dac"
Sep 09 16:34:54.110393 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:54.110356    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:34:54.287745 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:54.287706    2567 generic.go:356] "Generic (PLEG): container finished" podID="e8dedd14-b232-4636-9f89-8552b16c6d94" containerID="884d2a0324ebb757743bfd0e8dbcab01c8fec602b362586fef48e28e83bc693f" exitCode=0
Sep 09 16:34:54.287916 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:54.287755    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9" event={"ID":"e8dedd14-b232-4636-9f89-8552b16c6d94","Type":"ContainerDied","Data":"884d2a0324ebb757743bfd0e8dbcab01c8fec602b362586fef48e28e83bc693f"}
Sep 09 16:34:54.288300 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:54.288278    2567 scope.go:117] "RemoveContainer" containerID="884d2a0324ebb757743bfd0e8dbcab01c8fec602b362586fef48e28e83bc693f"
Sep 09 16:34:54.289870 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:54.289824    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:34:54.289997 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:54.289942    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:34:54.290067 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:54.290008    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:34:55.298397 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:55.298353    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9" event={"ID":"e8dedd14-b232-4636-9f89-8552b16c6d94","Type":"ContainerStarted","Data":"584c24ee282af8c241615283f570e3b9ff14a08378a3a2740f1e1b66d427fe45"}
Sep 09 16:34:56.088633 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:56.088597    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9"
Sep 09 16:34:56.089904 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:56.089868    2567 patch_prober.go:28] interesting pod/e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Sep 09 16:34:56.090060 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:56.089929    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9" podUID="e8dedd14-b232-4636-9f89-8552b16c6d94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Sep 09 16:34:56.788325 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:56.788289    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8"]
Sep 09 16:34:56.788749 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:56.788716    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae0afbee-c912-40f1-8e93-3db00cde2595" containerName="istio-proxy"
Sep 09 16:34:56.788749 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:56.788728    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae0afbee-c912-40f1-8e93-3db00cde2595" containerName="istio-proxy"
Sep 09 16:34:56.796885 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:56.796179    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8"
Sep 09 16:34:56.809738 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:56.801238    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8"]
Sep 09 16:34:56.809738 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:56.804799    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-ab78ef-openshift-default-dockercfg-bl6kg\""
Sep 09 16:34:56.895066 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:56.895022    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c864e699-e078-4530-bc10-da7da9e6cc0e-istio-podinfo\") pod \"e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8\" (UID: \"c864e699-e078-4530-bc10-da7da9e6cc0e\") " pod="openshift-ingress/e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8"
Sep 09 16:34:56.895678 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:56.895654    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c864e699-e078-4530-bc10-da7da9e6cc0e-istio-data\") pod \"e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8\" (UID: \"c864e699-e078-4530-bc10-da7da9e6cc0e\") " pod="openshift-ingress/e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8"
Sep 09 16:34:56.895959 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:56.895940    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c864e699-e078-4530-bc10-da7da9e6cc0e-istiod-ca-cert\") pod \"e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8\" (UID: \"c864e699-e078-4530-bc10-da7da9e6cc0e\") " pod="openshift-ingress/e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8"
Sep 09 16:34:56.896148 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:56.896132    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c864e699-e078-4530-bc10-da7da9e6cc0e-workload-socket\") pod \"e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8\" (UID: \"c864e699-e078-4530-bc10-da7da9e6cc0e\") " pod="openshift-ingress/e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8"
Sep 09 16:34:56.896324 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:56.896308    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c864e699-e078-4530-bc10-da7da9e6cc0e-istio-token\") pod \"e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8\" (UID: \"c864e699-e078-4530-bc10-da7da9e6cc0e\") " pod="openshift-ingress/e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8"
Sep 09 16:34:56.896510 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:56.896492    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c864e699-e078-4530-bc10-da7da9e6cc0e-credential-socket\") pod \"e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8\" (UID: \"c864e699-e078-4530-bc10-da7da9e6cc0e\") " pod="openshift-ingress/e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8"
Sep 09 16:34:56.896714 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:56.896698    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c864e699-e078-4530-bc10-da7da9e6cc0e-workload-certs\") pod \"e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8\" (UID: \"c864e699-e078-4530-bc10-da7da9e6cc0e\") " pod="openshift-ingress/e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8"
Sep 09 16:34:56.896889 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:56.896873    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lr2sx\" (UniqueName: \"kubernetes.io/projected/c864e699-e078-4530-bc10-da7da9e6cc0e-kube-api-access-lr2sx\") pod \"e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8\" (UID: \"c864e699-e078-4530-bc10-da7da9e6cc0e\") " pod="openshift-ingress/e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8"
Sep 09 16:34:56.897059 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:56.897043    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c864e699-e078-4530-bc10-da7da9e6cc0e-istio-envoy\") pod \"e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8\" (UID: \"c864e699-e078-4530-bc10-da7da9e6cc0e\") " pod="openshift-ingress/e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8"
Sep 09 16:34:56.998594 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:56.998547    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c864e699-e078-4530-bc10-da7da9e6cc0e-istiod-ca-cert\") pod \"e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8\" (UID: \"c864e699-e078-4530-bc10-da7da9e6cc0e\") " pod="openshift-ingress/e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8"
Sep 09 16:34:56.998825 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:56.998607    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c864e699-e078-4530-bc10-da7da9e6cc0e-workload-socket\") pod \"e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8\" (UID: \"c864e699-e078-4530-bc10-da7da9e6cc0e\") " pod="openshift-ingress/e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8"
Sep 09 16:34:56.998825 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:56.998638    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c864e699-e078-4530-bc10-da7da9e6cc0e-istio-token\") pod \"e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8\" (UID: \"c864e699-e078-4530-bc10-da7da9e6cc0e\") " pod="openshift-ingress/e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8"
Sep 09 16:34:56.998825 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:56.998669    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c864e699-e078-4530-bc10-da7da9e6cc0e-credential-socket\") pod \"e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8\" (UID: \"c864e699-e078-4530-bc10-da7da9e6cc0e\") " pod="openshift-ingress/e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8"
Sep 09 16:34:56.998825 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:56.998721    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c864e699-e078-4530-bc10-da7da9e6cc0e-workload-certs\") pod \"e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8\" (UID: \"c864e699-e078-4530-bc10-da7da9e6cc0e\") " pod="openshift-ingress/e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8"
Sep 09 16:34:56.998825 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:56.998763    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lr2sx\" (UniqueName: \"kubernetes.io/projected/c864e699-e078-4530-bc10-da7da9e6cc0e-kube-api-access-lr2sx\") pod \"e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8\" (UID: \"c864e699-e078-4530-bc10-da7da9e6cc0e\") " pod="openshift-ingress/e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8"
Sep 09 16:34:56.998825 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:56.998801    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c864e699-e078-4530-bc10-da7da9e6cc0e-istio-envoy\") pod \"e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8\" (UID: \"c864e699-e078-4530-bc10-da7da9e6cc0e\") " pod="openshift-ingress/e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8"
Sep 09 16:34:56.999182 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:56.998882    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c864e699-e078-4530-bc10-da7da9e6cc0e-istio-podinfo\") pod \"e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8\" (UID: \"c864e699-e078-4530-bc10-da7da9e6cc0e\") " pod="openshift-ingress/e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8"
Sep 09 16:34:56.999182 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:56.998915    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c864e699-e078-4530-bc10-da7da9e6cc0e-istio-data\") pod \"e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8\" (UID: \"c864e699-e078-4530-bc10-da7da9e6cc0e\") " pod="openshift-ingress/e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8"
Sep 09 16:34:56.999402 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:56.999374    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c864e699-e078-4530-bc10-da7da9e6cc0e-istio-data\") pod \"e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8\" (UID: \"c864e699-e078-4530-bc10-da7da9e6cc0e\") " pod="openshift-ingress/e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8"
Sep 09 16:34:57.000077 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:57.000049    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c864e699-e078-4530-bc10-da7da9e6cc0e-istiod-ca-cert\") pod \"e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8\" (UID: \"c864e699-e078-4530-bc10-da7da9e6cc0e\") " pod="openshift-ingress/e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8"
Sep 09 16:34:57.000315 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:57.000293    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c864e699-e078-4530-bc10-da7da9e6cc0e-workload-socket\") pod \"e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8\" (UID: \"c864e699-e078-4530-bc10-da7da9e6cc0e\") " pod="openshift-ingress/e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8"
Sep 09 16:34:57.000915 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:57.000887    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c864e699-e078-4530-bc10-da7da9e6cc0e-credential-socket\") pod \"e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8\" (UID: \"c864e699-e078-4530-bc10-da7da9e6cc0e\") " pod="openshift-ingress/e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8"
Sep 09 16:34:57.001156 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:57.001129    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c864e699-e078-4530-bc10-da7da9e6cc0e-workload-certs\") pod \"e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8\" (UID: \"c864e699-e078-4530-bc10-da7da9e6cc0e\") " pod="openshift-ingress/e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8"
Sep 09 16:34:57.013486 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:57.008335    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c864e699-e078-4530-bc10-da7da9e6cc0e-istio-podinfo\") pod \"e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8\" (UID: \"c864e699-e078-4530-bc10-da7da9e6cc0e\") " pod="openshift-ingress/e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8"
Sep 09 16:34:57.015959 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:57.015930    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c864e699-e078-4530-bc10-da7da9e6cc0e-istio-token\") pod \"e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8\" (UID: \"c864e699-e078-4530-bc10-da7da9e6cc0e\") " pod="openshift-ingress/e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8"
Sep 09 16:34:57.016158 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:57.016138    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c864e699-e078-4530-bc10-da7da9e6cc0e-istio-envoy\") pod \"e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8\" (UID: \"c864e699-e078-4530-bc10-da7da9e6cc0e\") " pod="openshift-ingress/e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8"
Sep 09 16:34:57.016284 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:57.016191    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lr2sx\" (UniqueName: \"kubernetes.io/projected/c864e699-e078-4530-bc10-da7da9e6cc0e-kube-api-access-lr2sx\") pod \"e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8\" (UID: \"c864e699-e078-4530-bc10-da7da9e6cc0e\") " pod="openshift-ingress/e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8"
Sep 09 16:34:57.088629 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:57.088592    2567 patch_prober.go:28] interesting pod/e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Sep 09 16:34:57.088799 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:57.088653    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9" podUID="e8dedd14-b232-4636-9f89-8552b16c6d94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Sep 09 16:34:57.128533 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:57.128498    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8"
Sep 09 16:34:57.404291 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:57.404189    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8"]
Sep 09 16:34:57.415480 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:34:57.409337    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc864e699_e078_4530_bc10_da7da9e6cc0e.slice/crio-bb15c0f59928820c3fb0abf883f0b20dfa039e92e4ce9b02f8efc12f85a428ae WatchSource:0}: Error finding container bb15c0f59928820c3fb0abf883f0b20dfa039e92e4ce9b02f8efc12f85a428ae: Status 404 returned error can't find the container with id bb15c0f59928820c3fb0abf883f0b20dfa039e92e4ce9b02f8efc12f85a428ae
Sep 09 16:34:57.415480 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:57.412539    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:34:57.415480 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:57.412627    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:34:57.415480 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:57.412671    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:34:58.088828 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:58.088791    2567 patch_prober.go:28] interesting pod/e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Sep 09 16:34:58.089236 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:58.088849    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9" podUID="e8dedd14-b232-4636-9f89-8552b16c6d94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Sep 09 16:34:58.323688 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:58.323652    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8" event={"ID":"c864e699-e078-4530-bc10-da7da9e6cc0e","Type":"ContainerStarted","Data":"9789f34aadf1447209581745e02a80845903a2987068fa4d79b62ebc47434fbe"}
Sep 09 16:34:58.323880 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:58.323705    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8" event={"ID":"c864e699-e078-4530-bc10-da7da9e6cc0e","Type":"ContainerStarted","Data":"bb15c0f59928820c3fb0abf883f0b20dfa039e92e4ce9b02f8efc12f85a428ae"}
Sep 09 16:34:58.342430 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:58.342329    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8" podStartSLOduration=2.3423146360000002 podStartE2EDuration="2.342314636s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 16:34:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 16:34:58.340976126 +0000 UTC m=+1573.758826126" watchObservedRunningTime="2026-09-09 16:34:58.342314636 +0000 UTC m=+1573.760164680"
Sep 09 16:34:59.088707 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:59.088663    2567 patch_prober.go:28] interesting pod/e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Sep 09 16:34:59.088897 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:59.088726    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9" podUID="e8dedd14-b232-4636-9f89-8552b16c6d94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Sep 09 16:34:59.129634 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:59.129589    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8"
Sep 09 16:34:59.135039 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:59.135010    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8"
Sep 09 16:34:59.330056 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:59.330031    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8"
Sep 09 16:34:59.331200 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:34:59.331178    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8"
Sep 09 16:35:00.089050 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:00.089016    2567 patch_prober.go:28] interesting pod/e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Sep 09 16:35:00.089449 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:00.089068    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9" podUID="e8dedd14-b232-4636-9f89-8552b16c6d94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Sep 09 16:35:00.157354 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:00.157321    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9"
Sep 09 16:35:01.092871 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:01.092843    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9"
Sep 09 16:35:01.093965 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:01.093947    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9"
Sep 09 16:35:01.530125 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:01.530049    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ab78ef-6b5b97c67b-pz5ln"]
Sep 09 16:35:01.534802 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:01.534779    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab78ef-6b5b97c67b-pz5ln"
Sep 09 16:35:01.537063 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:01.537035    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-ab78ef-serving-cert\""
Sep 09 16:35:01.542238 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:01.542213    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ab78ef-6b5b97c67b-pz5ln"]
Sep 09 16:35:01.659674 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:01.659607    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jngxq\" (UniqueName: \"kubernetes.io/projected/50995643-35be-45c9-b756-280a7d978dc3-kube-api-access-jngxq\") pod \"maas-api-e2e-api-b-ab78ef-6b5b97c67b-pz5ln\" (UID: \"50995643-35be-45c9-b756-280a7d978dc3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab78ef-6b5b97c67b-pz5ln"
Sep 09 16:35:01.659674 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:01.659673    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50995643-35be-45c9-b756-280a7d978dc3-maas-api-tls\") pod \"maas-api-e2e-api-b-ab78ef-6b5b97c67b-pz5ln\" (UID: \"50995643-35be-45c9-b756-280a7d978dc3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab78ef-6b5b97c67b-pz5ln"
Sep 09 16:35:01.659960 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:01.659725    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50995643-35be-45c9-b756-280a7d978dc3-openshift-service-ca\") pod \"maas-api-e2e-api-b-ab78ef-6b5b97c67b-pz5ln\" (UID: \"50995643-35be-45c9-b756-280a7d978dc3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab78ef-6b5b97c67b-pz5ln"
Sep 09 16:35:01.659960 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:01.659769    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50995643-35be-45c9-b756-280a7d978dc3-metrics-certs\") pod \"maas-api-e2e-api-b-ab78ef-6b5b97c67b-pz5ln\" (UID: \"50995643-35be-45c9-b756-280a7d978dc3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab78ef-6b5b97c67b-pz5ln"
Sep 09 16:35:01.760921 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:01.760881    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jngxq\" (UniqueName: \"kubernetes.io/projected/50995643-35be-45c9-b756-280a7d978dc3-kube-api-access-jngxq\") pod \"maas-api-e2e-api-b-ab78ef-6b5b97c67b-pz5ln\" (UID: \"50995643-35be-45c9-b756-280a7d978dc3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab78ef-6b5b97c67b-pz5ln"
Sep 09 16:35:01.760921 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:01.760930    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50995643-35be-45c9-b756-280a7d978dc3-maas-api-tls\") pod \"maas-api-e2e-api-b-ab78ef-6b5b97c67b-pz5ln\" (UID: \"50995643-35be-45c9-b756-280a7d978dc3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab78ef-6b5b97c67b-pz5ln"
Sep 09 16:35:01.761194 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:01.760961    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50995643-35be-45c9-b756-280a7d978dc3-openshift-service-ca\") pod \"maas-api-e2e-api-b-ab78ef-6b5b97c67b-pz5ln\" (UID: \"50995643-35be-45c9-b756-280a7d978dc3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab78ef-6b5b97c67b-pz5ln"
Sep 09 16:35:01.761194 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:01.760985    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50995643-35be-45c9-b756-280a7d978dc3-metrics-certs\") pod \"maas-api-e2e-api-b-ab78ef-6b5b97c67b-pz5ln\" (UID: \"50995643-35be-45c9-b756-280a7d978dc3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab78ef-6b5b97c67b-pz5ln"
Sep 09 16:35:01.761194 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:35:01.761056    2567 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-ab78ef-serving-cert: secret "maas-api-e2e-api-b-ab78ef-serving-cert" not found
Sep 09 16:35:01.761194 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:35:01.761135    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50995643-35be-45c9-b756-280a7d978dc3-maas-api-tls podName:50995643-35be-45c9-b756-280a7d978dc3 nodeName:}" failed. No retries permitted until 2026-09-09 16:35:02.261115455 +0000 UTC m=+1577.678965448 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/50995643-35be-45c9-b756-280a7d978dc3-maas-api-tls") pod "maas-api-e2e-api-b-ab78ef-6b5b97c67b-pz5ln" (UID: "50995643-35be-45c9-b756-280a7d978dc3") : secret "maas-api-e2e-api-b-ab78ef-serving-cert" not found
Sep 09 16:35:01.761679 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:01.761658    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50995643-35be-45c9-b756-280a7d978dc3-openshift-service-ca\") pod \"maas-api-e2e-api-b-ab78ef-6b5b97c67b-pz5ln\" (UID: \"50995643-35be-45c9-b756-280a7d978dc3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab78ef-6b5b97c67b-pz5ln"
Sep 09 16:35:01.763720 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:01.763696    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50995643-35be-45c9-b756-280a7d978dc3-metrics-certs\") pod \"maas-api-e2e-api-b-ab78ef-6b5b97c67b-pz5ln\" (UID: \"50995643-35be-45c9-b756-280a7d978dc3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab78ef-6b5b97c67b-pz5ln"
Sep 09 16:35:01.772989 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:01.772965    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jngxq\" (UniqueName: \"kubernetes.io/projected/50995643-35be-45c9-b756-280a7d978dc3-kube-api-access-jngxq\") pod \"maas-api-e2e-api-b-ab78ef-6b5b97c67b-pz5ln\" (UID: \"50995643-35be-45c9-b756-280a7d978dc3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab78ef-6b5b97c67b-pz5ln"
Sep 09 16:35:02.266298 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:02.266254    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50995643-35be-45c9-b756-280a7d978dc3-maas-api-tls\") pod \"maas-api-e2e-api-b-ab78ef-6b5b97c67b-pz5ln\" (UID: \"50995643-35be-45c9-b756-280a7d978dc3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab78ef-6b5b97c67b-pz5ln"
Sep 09 16:35:02.268925 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:02.268881    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50995643-35be-45c9-b756-280a7d978dc3-maas-api-tls\") pod \"maas-api-e2e-api-b-ab78ef-6b5b97c67b-pz5ln\" (UID: \"50995643-35be-45c9-b756-280a7d978dc3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab78ef-6b5b97c67b-pz5ln"
Sep 09 16:35:02.448780 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:02.448742    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab78ef-6b5b97c67b-pz5ln"
Sep 09 16:35:02.720694 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:02.720670    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ab78ef-6b5b97c67b-pz5ln"]
Sep 09 16:35:02.723827 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:35:02.723801    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50995643_35be_45c9_b756_280a7d978dc3.slice/crio-1eb4e5833284d9b62be52e04f21a7887b0f823aeb017406953c28bce3b09e8b0 WatchSource:0}: Error finding container 1eb4e5833284d9b62be52e04f21a7887b0f823aeb017406953c28bce3b09e8b0: Status 404 returned error can't find the container with id 1eb4e5833284d9b62be52e04f21a7887b0f823aeb017406953c28bce3b09e8b0
Sep 09 16:35:03.359441 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:03.359404    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab78ef-6b5b97c67b-pz5ln" event={"ID":"50995643-35be-45c9-b756-280a7d978dc3","Type":"ContainerStarted","Data":"c26dc2642d7c9c154cbe76c1a2830402afdd19ea7a1147c7c1e0bc9f7202258c"}
Sep 09 16:35:03.359860 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:03.359603    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab78ef-6b5b97c67b-pz5ln"
Sep 09 16:35:03.359860 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:03.359625    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab78ef-6b5b97c67b-pz5ln" event={"ID":"50995643-35be-45c9-b756-280a7d978dc3","Type":"ContainerStarted","Data":"1eb4e5833284d9b62be52e04f21a7887b0f823aeb017406953c28bce3b09e8b0"}
Sep 09 16:35:03.377485 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:03.377412    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab78ef-6b5b97c67b-pz5ln" podStartSLOduration=1.935522264 podStartE2EDuration="2.377396469s" totalImagesPullingTime="441.874205ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 16:35:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 16:35:03.374824692 +0000 UTC m=+1578.792674695" watchObservedRunningTime="2026-09-09 16:35:03.377396469 +0000 UTC m=+1578.795246469"
Sep 09 16:35:09.370571 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:09.370530    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab78ef-6b5b97c67b-pz5ln"
Sep 09 16:35:09.912798 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:09.912757    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:35:20.464081 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:20.464047    2567 generic.go:356] "Generic (PLEG): container finished" podID="c864e699-e078-4530-bc10-da7da9e6cc0e" containerID="9789f34aadf1447209581745e02a80845903a2987068fa4d79b62ebc47434fbe" exitCode=0
Sep 09 16:35:20.464525 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:20.464116    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8" event={"ID":"c864e699-e078-4530-bc10-da7da9e6cc0e","Type":"ContainerDied","Data":"9789f34aadf1447209581745e02a80845903a2987068fa4d79b62ebc47434fbe"}
Sep 09 16:35:20.464572 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:20.464533    2567 scope.go:117] "RemoveContainer" containerID="9789f34aadf1447209581745e02a80845903a2987068fa4d79b62ebc47434fbe"
Sep 09 16:35:20.465669 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:20.465641    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:35:20.465742 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:20.465712    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:35:20.465778 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:20.465746    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:35:21.472338 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:21.472296    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8" event={"ID":"c864e699-e078-4530-bc10-da7da9e6cc0e","Type":"ContainerStarted","Data":"a663b574ba25c79ae554a16ca5665c9949f3f743386031d2ca7331d8ead81116"}
Sep 09 16:35:22.129435 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:22.129386    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8"
Sep 09 16:35:22.129705 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:22.129679    2567 patch_prober.go:28] interesting pod/e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Sep 09 16:35:22.129813 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:22.129780    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8" podUID="c864e699-e078-4530-bc10-da7da9e6cc0e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Sep 09 16:35:23.129434 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:23.129391    2567 patch_prober.go:28] interesting pod/e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Sep 09 16:35:23.129958 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:23.129493    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8" podUID="c864e699-e078-4530-bc10-da7da9e6cc0e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Sep 09 16:35:23.403286 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:23.403178    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:35:24.129723 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:24.129688    2567 patch_prober.go:28] interesting pod/e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Sep 09 16:35:24.130147 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:24.129745    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8" podUID="c864e699-e078-4530-bc10-da7da9e6cc0e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Sep 09 16:35:25.129776 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:25.129741    2567 patch_prober.go:28] interesting pod/e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Sep 09 16:35:25.130173 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:25.129795    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8" podUID="c864e699-e078-4530-bc10-da7da9e6cc0e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Sep 09 16:35:26.129936 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:26.129894    2567 patch_prober.go:28] interesting pod/e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Sep 09 16:35:26.130346 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:26.129962    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8" podUID="c864e699-e078-4530-bc10-da7da9e6cc0e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Sep 09 16:35:27.133160 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:27.133127    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8"
Sep 09 16:35:27.133654 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:27.133537    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8"
Sep 09 16:35:27.134285 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:27.134264    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8"
Sep 09 16:35:45.237666 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:45.237575    2567 scope.go:117] "RemoveContainer" containerID="2a9de9e1b66af8ea70047b16a9d4a8c2d425c9b0adc0297bdb28db62252349ce"
Sep 09 16:35:45.246022 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:45.245996    2567 scope.go:117] "RemoveContainer" containerID="528f4d915cf01263331bfbabb99a58410e059bb8f5d1bf8457abb5aff08ded26"
Sep 09 16:35:48.709163 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:48.709123    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:35:50.501614 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:50.501567    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9"]
Sep 09 16:35:50.502030 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:50.501893    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9" podUID="e8dedd14-b232-4636-9f89-8552b16c6d94" containerName="istio-proxy" containerID="cri-o://584c24ee282af8c241615283f570e3b9ff14a08378a3a2740f1e1b66d427fe45" gracePeriod=30
Sep 09 16:35:55.663606 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:55.663568    2567 generic.go:356] "Generic (PLEG): container finished" podID="e8dedd14-b232-4636-9f89-8552b16c6d94" containerID="584c24ee282af8c241615283f570e3b9ff14a08378a3a2740f1e1b66d427fe45" exitCode=0
Sep 09 16:35:55.664081 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:55.663642    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9" event={"ID":"e8dedd14-b232-4636-9f89-8552b16c6d94","Type":"ContainerDied","Data":"584c24ee282af8c241615283f570e3b9ff14a08378a3a2740f1e1b66d427fe45"}
Sep 09 16:35:55.664081 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:55.663726    2567 scope.go:117] "RemoveContainer" containerID="884d2a0324ebb757743bfd0e8dbcab01c8fec602b362586fef48e28e83bc693f"
Sep 09 16:35:55.773522 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:55.770624    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9"
Sep 09 16:35:55.837006 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:55.836970    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8dedd14-b232-4636-9f89-8552b16c6d94-istio-token\") pod \"e8dedd14-b232-4636-9f89-8552b16c6d94\" (UID: \"e8dedd14-b232-4636-9f89-8552b16c6d94\") "
Sep 09 16:35:55.837006 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:55.837007    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8dedd14-b232-4636-9f89-8552b16c6d94-istio-podinfo\") pod \"e8dedd14-b232-4636-9f89-8552b16c6d94\" (UID: \"e8dedd14-b232-4636-9f89-8552b16c6d94\") "
Sep 09 16:35:55.837209 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:55.837046    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8dedd14-b232-4636-9f89-8552b16c6d94-credential-socket\") pod \"e8dedd14-b232-4636-9f89-8552b16c6d94\" (UID: \"e8dedd14-b232-4636-9f89-8552b16c6d94\") "
Sep 09 16:35:55.837209 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:55.837138    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8dedd14-b232-4636-9f89-8552b16c6d94-workload-socket\") pod \"e8dedd14-b232-4636-9f89-8552b16c6d94\" (UID: \"e8dedd14-b232-4636-9f89-8552b16c6d94\") "
Sep 09 16:35:55.837209 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:55.837177    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8dedd14-b232-4636-9f89-8552b16c6d94-workload-certs\") pod \"e8dedd14-b232-4636-9f89-8552b16c6d94\" (UID: \"e8dedd14-b232-4636-9f89-8552b16c6d94\") "
Sep 09 16:35:55.837370 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:55.837217    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jh9jg\" (UniqueName: \"kubernetes.io/projected/e8dedd14-b232-4636-9f89-8552b16c6d94-kube-api-access-jh9jg\") pod \"e8dedd14-b232-4636-9f89-8552b16c6d94\" (UID: \"e8dedd14-b232-4636-9f89-8552b16c6d94\") "
Sep 09 16:35:55.837370 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:55.837304    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8dedd14-b232-4636-9f89-8552b16c6d94-istio-data\") pod \"e8dedd14-b232-4636-9f89-8552b16c6d94\" (UID: \"e8dedd14-b232-4636-9f89-8552b16c6d94\") "
Sep 09 16:35:55.837370 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:55.837328    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8dedd14-b232-4636-9f89-8552b16c6d94-istiod-ca-cert\") pod \"e8dedd14-b232-4636-9f89-8552b16c6d94\" (UID: \"e8dedd14-b232-4636-9f89-8552b16c6d94\") "
Sep 09 16:35:55.837370 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:55.837360    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8dedd14-b232-4636-9f89-8552b16c6d94-istio-envoy\") pod \"e8dedd14-b232-4636-9f89-8552b16c6d94\" (UID: \"e8dedd14-b232-4636-9f89-8552b16c6d94\") "
Sep 09 16:35:55.837615 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:55.837406    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8dedd14-b232-4636-9f89-8552b16c6d94-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e8dedd14-b232-4636-9f89-8552b16c6d94" (UID: "e8dedd14-b232-4636-9f89-8552b16c6d94"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:35:55.837615 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:55.837420    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8dedd14-b232-4636-9f89-8552b16c6d94-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e8dedd14-b232-4636-9f89-8552b16c6d94" (UID: "e8dedd14-b232-4636-9f89-8552b16c6d94"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:35:55.837615 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:55.837515    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8dedd14-b232-4636-9f89-8552b16c6d94-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e8dedd14-b232-4636-9f89-8552b16c6d94" (UID: "e8dedd14-b232-4636-9f89-8552b16c6d94"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:35:55.837762 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:55.837676    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8dedd14-b232-4636-9f89-8552b16c6d94-istio-data" (OuterVolumeSpecName: "istio-data") pod "e8dedd14-b232-4636-9f89-8552b16c6d94" (UID: "e8dedd14-b232-4636-9f89-8552b16c6d94"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:35:55.837909 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:55.837887    2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8dedd14-b232-4636-9f89-8552b16c6d94-istio-data\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:35:55.837995 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:55.837915    2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8dedd14-b232-4636-9f89-8552b16c6d94-credential-socket\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:35:55.837995 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:55.837931    2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8dedd14-b232-4636-9f89-8552b16c6d94-workload-socket\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:35:55.837995 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:55.837929    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8dedd14-b232-4636-9f89-8552b16c6d94-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e8dedd14-b232-4636-9f89-8552b16c6d94" (UID: "e8dedd14-b232-4636-9f89-8552b16c6d94"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 16:35:55.837995 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:55.837944    2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8dedd14-b232-4636-9f89-8552b16c6d94-workload-certs\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:35:55.839498 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:55.839469    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8dedd14-b232-4636-9f89-8552b16c6d94-istio-token" (OuterVolumeSpecName: "istio-token") pod "e8dedd14-b232-4636-9f89-8552b16c6d94" (UID: "e8dedd14-b232-4636-9f89-8552b16c6d94"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 16:35:55.839613 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:55.839519    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e8dedd14-b232-4636-9f89-8552b16c6d94-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e8dedd14-b232-4636-9f89-8552b16c6d94" (UID: "e8dedd14-b232-4636-9f89-8552b16c6d94"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 16:35:55.839702 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:55.839683    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8dedd14-b232-4636-9f89-8552b16c6d94-kube-api-access-jh9jg" (OuterVolumeSpecName: "kube-api-access-jh9jg") pod "e8dedd14-b232-4636-9f89-8552b16c6d94" (UID: "e8dedd14-b232-4636-9f89-8552b16c6d94"). InnerVolumeSpecName "kube-api-access-jh9jg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 16:35:55.839816 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:55.839802    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8dedd14-b232-4636-9f89-8552b16c6d94-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e8dedd14-b232-4636-9f89-8552b16c6d94" (UID: "e8dedd14-b232-4636-9f89-8552b16c6d94"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:35:55.938911 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:55.938872    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jh9jg\" (UniqueName: \"kubernetes.io/projected/e8dedd14-b232-4636-9f89-8552b16c6d94-kube-api-access-jh9jg\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:35:55.938911 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:55.938904    2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8dedd14-b232-4636-9f89-8552b16c6d94-istiod-ca-cert\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:35:55.938911 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:55.938914    2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8dedd14-b232-4636-9f89-8552b16c6d94-istio-envoy\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:35:55.938911 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:55.938922    2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8dedd14-b232-4636-9f89-8552b16c6d94-istio-token\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:35:55.939192 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:55.938930    2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8dedd14-b232-4636-9f89-8552b16c6d94-istio-podinfo\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:35:56.671019 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:56.670991    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9"
Sep 09 16:35:56.671427 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:56.671043    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9" event={"ID":"e8dedd14-b232-4636-9f89-8552b16c6d94","Type":"ContainerDied","Data":"7f31c3a6277e2cc34db6c174a3ffb0b351e56d38252b828e6de003d509be2902"}
Sep 09 16:35:56.671427 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:56.671096    2567 scope.go:117] "RemoveContainer" containerID="584c24ee282af8c241615283f570e3b9ff14a08378a3a2740f1e1b66d427fe45"
Sep 09 16:35:56.692546 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:56.692517    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9"]
Sep 09 16:35:56.697731 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:56.697709    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-5d8327-openshift-default-64c78986cd-24ch9"]
Sep 09 16:35:57.036924 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:35:57.036846    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8dedd14-b232-4636-9f89-8552b16c6d94" path="/var/lib/kubelet/pods/e8dedd14-b232-4636-9f89-8552b16c6d94/volumes"
Sep 09 16:36:03.710048 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:03.710008    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:36:06.329533 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:06.329494    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ab78ef-6b5b97c67b-pz5ln"]
Sep 09 16:36:06.329987 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:06.329827    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab78ef-6b5b97c67b-pz5ln" podUID="50995643-35be-45c9-b756-280a7d978dc3" containerName="maas-api" containerID="cri-o://c26dc2642d7c9c154cbe76c1a2830402afdd19ea7a1147c7c1e0bc9f7202258c" gracePeriod=30
Sep 09 16:36:06.597368 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:06.597298    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab78ef-6b5b97c67b-pz5ln"
Sep 09 16:36:06.733882 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:06.733842    2567 generic.go:356] "Generic (PLEG): container finished" podID="50995643-35be-45c9-b756-280a7d978dc3" containerID="c26dc2642d7c9c154cbe76c1a2830402afdd19ea7a1147c7c1e0bc9f7202258c" exitCode=0
Sep 09 16:36:06.734092 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:06.733909    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab78ef-6b5b97c67b-pz5ln" event={"ID":"50995643-35be-45c9-b756-280a7d978dc3","Type":"ContainerDied","Data":"c26dc2642d7c9c154cbe76c1a2830402afdd19ea7a1147c7c1e0bc9f7202258c"}
Sep 09 16:36:06.734092 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:06.733918    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab78ef-6b5b97c67b-pz5ln"
Sep 09 16:36:06.734092 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:06.733991    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab78ef-6b5b97c67b-pz5ln" event={"ID":"50995643-35be-45c9-b756-280a7d978dc3","Type":"ContainerDied","Data":"1eb4e5833284d9b62be52e04f21a7887b0f823aeb017406953c28bce3b09e8b0"}
Sep 09 16:36:06.734092 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:06.734022    2567 scope.go:117] "RemoveContainer" containerID="c26dc2642d7c9c154cbe76c1a2830402afdd19ea7a1147c7c1e0bc9f7202258c"
Sep 09 16:36:06.751301 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:06.749772    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50995643-35be-45c9-b756-280a7d978dc3-openshift-service-ca\") pod \"50995643-35be-45c9-b756-280a7d978dc3\" (UID: \"50995643-35be-45c9-b756-280a7d978dc3\") "
Sep 09 16:36:06.751301 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:06.749840    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50995643-35be-45c9-b756-280a7d978dc3-metrics-certs\") pod \"50995643-35be-45c9-b756-280a7d978dc3\" (UID: \"50995643-35be-45c9-b756-280a7d978dc3\") "
Sep 09 16:36:06.751301 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:06.750004    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jngxq\" (UniqueName: \"kubernetes.io/projected/50995643-35be-45c9-b756-280a7d978dc3-kube-api-access-jngxq\") pod \"50995643-35be-45c9-b756-280a7d978dc3\" (UID: \"50995643-35be-45c9-b756-280a7d978dc3\") "
Sep 09 16:36:06.751301 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:06.750072    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50995643-35be-45c9-b756-280a7d978dc3-maas-api-tls\") pod \"50995643-35be-45c9-b756-280a7d978dc3\" (UID: \"50995643-35be-45c9-b756-280a7d978dc3\") "
Sep 09 16:36:06.751301 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:06.751010    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50995643-35be-45c9-b756-280a7d978dc3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "50995643-35be-45c9-b756-280a7d978dc3" (UID: "50995643-35be-45c9-b756-280a7d978dc3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 16:36:06.752042 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:06.752018    2567 scope.go:117] "RemoveContainer" containerID="c26dc2642d7c9c154cbe76c1a2830402afdd19ea7a1147c7c1e0bc9f7202258c"
Sep 09 16:36:06.754380 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:36:06.752509    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c26dc2642d7c9c154cbe76c1a2830402afdd19ea7a1147c7c1e0bc9f7202258c\": container with ID starting with c26dc2642d7c9c154cbe76c1a2830402afdd19ea7a1147c7c1e0bc9f7202258c not found: ID does not exist" containerID="c26dc2642d7c9c154cbe76c1a2830402afdd19ea7a1147c7c1e0bc9f7202258c"
Sep 09 16:36:06.754380 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:06.752545    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c26dc2642d7c9c154cbe76c1a2830402afdd19ea7a1147c7c1e0bc9f7202258c"} err="failed to get container status \"c26dc2642d7c9c154cbe76c1a2830402afdd19ea7a1147c7c1e0bc9f7202258c\": rpc error: code = NotFound desc = could not find container \"c26dc2642d7c9c154cbe76c1a2830402afdd19ea7a1147c7c1e0bc9f7202258c\": container with ID starting with c26dc2642d7c9c154cbe76c1a2830402afdd19ea7a1147c7c1e0bc9f7202258c not found: ID does not exist"
Sep 09 16:36:06.758345 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:06.758281    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50995643-35be-45c9-b756-280a7d978dc3-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "50995643-35be-45c9-b756-280a7d978dc3" (UID: "50995643-35be-45c9-b756-280a7d978dc3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 16:36:06.760381 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:06.760320    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50995643-35be-45c9-b756-280a7d978dc3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "50995643-35be-45c9-b756-280a7d978dc3" (UID: "50995643-35be-45c9-b756-280a7d978dc3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 16:36:06.760590 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:06.760538    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50995643-35be-45c9-b756-280a7d978dc3-kube-api-access-jngxq" (OuterVolumeSpecName: "kube-api-access-jngxq") pod "50995643-35be-45c9-b756-280a7d978dc3" (UID: "50995643-35be-45c9-b756-280a7d978dc3"). InnerVolumeSpecName "kube-api-access-jngxq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 16:36:06.851646 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:06.851545    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jngxq\" (UniqueName: \"kubernetes.io/projected/50995643-35be-45c9-b756-280a7d978dc3-kube-api-access-jngxq\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:36:06.851646 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:06.851581    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50995643-35be-45c9-b756-280a7d978dc3-maas-api-tls\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:36:06.851646 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:06.851597    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50995643-35be-45c9-b756-280a7d978dc3-openshift-service-ca\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:36:06.851646 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:06.851609    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50995643-35be-45c9-b756-280a7d978dc3-metrics-certs\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:36:07.060369 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:07.060337    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ab78ef-6b5b97c67b-pz5ln"]
Sep 09 16:36:07.064345 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:07.064320    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ab78ef-6b5b97c67b-pz5ln"]
Sep 09 16:36:09.036943 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:09.036907    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50995643-35be-45c9-b756-280a7d978dc3" path="/var/lib/kubelet/pods/50995643-35be-45c9-b756-280a7d978dc3/volumes"
Sep 09 16:36:13.338603 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:13.338569    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8"]
Sep 09 16:36:13.339124 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:13.338800    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8" podUID="c864e699-e078-4530-bc10-da7da9e6cc0e" containerName="istio-proxy" containerID="cri-o://a663b574ba25c79ae554a16ca5665c9949f3f743386031d2ca7331d8ead81116" gracePeriod=30
Sep 09 16:36:18.600523 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:18.600434    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8"
Sep 09 16:36:18.665134 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:18.665098    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c864e699-e078-4530-bc10-da7da9e6cc0e-istio-data\") pod \"c864e699-e078-4530-bc10-da7da9e6cc0e\" (UID: \"c864e699-e078-4530-bc10-da7da9e6cc0e\") "
Sep 09 16:36:18.665327 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:18.665172    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lr2sx\" (UniqueName: \"kubernetes.io/projected/c864e699-e078-4530-bc10-da7da9e6cc0e-kube-api-access-lr2sx\") pod \"c864e699-e078-4530-bc10-da7da9e6cc0e\" (UID: \"c864e699-e078-4530-bc10-da7da9e6cc0e\") "
Sep 09 16:36:18.665327 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:18.665213    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c864e699-e078-4530-bc10-da7da9e6cc0e-workload-socket\") pod \"c864e699-e078-4530-bc10-da7da9e6cc0e\" (UID: \"c864e699-e078-4530-bc10-da7da9e6cc0e\") "
Sep 09 16:36:18.665327 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:18.665236    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c864e699-e078-4530-bc10-da7da9e6cc0e-workload-certs\") pod \"c864e699-e078-4530-bc10-da7da9e6cc0e\" (UID: \"c864e699-e078-4530-bc10-da7da9e6cc0e\") "
Sep 09 16:36:18.665327 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:18.665253    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c864e699-e078-4530-bc10-da7da9e6cc0e-istio-podinfo\") pod \"c864e699-e078-4530-bc10-da7da9e6cc0e\" (UID: \"c864e699-e078-4530-bc10-da7da9e6cc0e\") "
Sep 09 16:36:18.665327 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:18.665277    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c864e699-e078-4530-bc10-da7da9e6cc0e-istio-token\") pod \"c864e699-e078-4530-bc10-da7da9e6cc0e\" (UID: \"c864e699-e078-4530-bc10-da7da9e6cc0e\") "
Sep 09 16:36:18.665327 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:18.665313    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c864e699-e078-4530-bc10-da7da9e6cc0e-istio-envoy\") pod \"c864e699-e078-4530-bc10-da7da9e6cc0e\" (UID: \"c864e699-e078-4530-bc10-da7da9e6cc0e\") "
Sep 09 16:36:18.665607 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:18.665362    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c864e699-e078-4530-bc10-da7da9e6cc0e-credential-socket\") pod \"c864e699-e078-4530-bc10-da7da9e6cc0e\" (UID: \"c864e699-e078-4530-bc10-da7da9e6cc0e\") "
Sep 09 16:36:18.665607 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:18.665391    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c864e699-e078-4530-bc10-da7da9e6cc0e-istiod-ca-cert\") pod \"c864e699-e078-4530-bc10-da7da9e6cc0e\" (UID: \"c864e699-e078-4530-bc10-da7da9e6cc0e\") "
Sep 09 16:36:18.665607 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:18.665489    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c864e699-e078-4530-bc10-da7da9e6cc0e-istio-data" (OuterVolumeSpecName: "istio-data") pod "c864e699-e078-4530-bc10-da7da9e6cc0e" (UID: "c864e699-e078-4530-bc10-da7da9e6cc0e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:36:18.665607 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:18.665500    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c864e699-e078-4530-bc10-da7da9e6cc0e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c864e699-e078-4530-bc10-da7da9e6cc0e" (UID: "c864e699-e078-4530-bc10-da7da9e6cc0e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:36:18.665784 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:18.665721    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c864e699-e078-4530-bc10-da7da9e6cc0e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c864e699-e078-4530-bc10-da7da9e6cc0e" (UID: "c864e699-e078-4530-bc10-da7da9e6cc0e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:36:18.665843 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:18.665784    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c864e699-e078-4530-bc10-da7da9e6cc0e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c864e699-e078-4530-bc10-da7da9e6cc0e" (UID: "c864e699-e078-4530-bc10-da7da9e6cc0e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:36:18.665843 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:18.665829    2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c864e699-e078-4530-bc10-da7da9e6cc0e-credential-socket\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:36:18.665959 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:18.665849    2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c864e699-e078-4530-bc10-da7da9e6cc0e-istio-data\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:36:18.665959 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:18.665863    2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c864e699-e078-4530-bc10-da7da9e6cc0e-workload-socket\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:36:18.665959 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:18.665914    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c864e699-e078-4530-bc10-da7da9e6cc0e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c864e699-e078-4530-bc10-da7da9e6cc0e" (UID: "c864e699-e078-4530-bc10-da7da9e6cc0e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 16:36:18.667795 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:18.667765    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c864e699-e078-4530-bc10-da7da9e6cc0e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c864e699-e078-4530-bc10-da7da9e6cc0e" (UID: "c864e699-e078-4530-bc10-da7da9e6cc0e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:36:18.667795 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:18.667771    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c864e699-e078-4530-bc10-da7da9e6cc0e-istio-token" (OuterVolumeSpecName: "istio-token") pod "c864e699-e078-4530-bc10-da7da9e6cc0e" (UID: "c864e699-e078-4530-bc10-da7da9e6cc0e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 16:36:18.667974 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:18.667763    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c864e699-e078-4530-bc10-da7da9e6cc0e-kube-api-access-lr2sx" (OuterVolumeSpecName: "kube-api-access-lr2sx") pod "c864e699-e078-4530-bc10-da7da9e6cc0e" (UID: "c864e699-e078-4530-bc10-da7da9e6cc0e"). InnerVolumeSpecName "kube-api-access-lr2sx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 16:36:18.667974 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:18.667828    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c864e699-e078-4530-bc10-da7da9e6cc0e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c864e699-e078-4530-bc10-da7da9e6cc0e" (UID: "c864e699-e078-4530-bc10-da7da9e6cc0e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 16:36:18.766864 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:18.766827    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lr2sx\" (UniqueName: \"kubernetes.io/projected/c864e699-e078-4530-bc10-da7da9e6cc0e-kube-api-access-lr2sx\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:36:18.766864 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:18.766859    2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c864e699-e078-4530-bc10-da7da9e6cc0e-workload-certs\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:36:18.766864 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:18.766871    2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c864e699-e078-4530-bc10-da7da9e6cc0e-istio-podinfo\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:36:18.767133 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:18.766881    2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c864e699-e078-4530-bc10-da7da9e6cc0e-istio-token\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:36:18.767133 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:18.766889    2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c864e699-e078-4530-bc10-da7da9e6cc0e-istio-envoy\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:36:18.767133 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:18.766898    2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c864e699-e078-4530-bc10-da7da9e6cc0e-istiod-ca-cert\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:36:18.798951 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:18.798912    2567 generic.go:356] "Generic (PLEG): container finished" podID="c864e699-e078-4530-bc10-da7da9e6cc0e" containerID="a663b574ba25c79ae554a16ca5665c9949f3f743386031d2ca7331d8ead81116" exitCode=0
Sep 09 16:36:18.799094 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:18.798962    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8"
Sep 09 16:36:18.799094 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:18.798998    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8" event={"ID":"c864e699-e078-4530-bc10-da7da9e6cc0e","Type":"ContainerDied","Data":"a663b574ba25c79ae554a16ca5665c9949f3f743386031d2ca7331d8ead81116"}
Sep 09 16:36:18.799094 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:18.799053    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8" event={"ID":"c864e699-e078-4530-bc10-da7da9e6cc0e","Type":"ContainerDied","Data":"bb15c0f59928820c3fb0abf883f0b20dfa039e92e4ce9b02f8efc12f85a428ae"}
Sep 09 16:36:18.799094 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:18.799075    2567 scope.go:117] "RemoveContainer" containerID="a663b574ba25c79ae554a16ca5665c9949f3f743386031d2ca7331d8ead81116"
Sep 09 16:36:18.808673 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:18.808653    2567 scope.go:117] "RemoveContainer" containerID="9789f34aadf1447209581745e02a80845903a2987068fa4d79b62ebc47434fbe"
Sep 09 16:36:18.817015 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:18.816975    2567 scope.go:117] "RemoveContainer" containerID="a663b574ba25c79ae554a16ca5665c9949f3f743386031d2ca7331d8ead81116"
Sep 09 16:36:18.817713 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:36:18.817392    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a663b574ba25c79ae554a16ca5665c9949f3f743386031d2ca7331d8ead81116\": container with ID starting with a663b574ba25c79ae554a16ca5665c9949f3f743386031d2ca7331d8ead81116 not found: ID does not exist" containerID="a663b574ba25c79ae554a16ca5665c9949f3f743386031d2ca7331d8ead81116"
Sep 09 16:36:18.817713 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:18.817563    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a663b574ba25c79ae554a16ca5665c9949f3f743386031d2ca7331d8ead81116"} err="failed to get container status \"a663b574ba25c79ae554a16ca5665c9949f3f743386031d2ca7331d8ead81116\": rpc error: code = NotFound desc = could not find container \"a663b574ba25c79ae554a16ca5665c9949f3f743386031d2ca7331d8ead81116\": container with ID starting with a663b574ba25c79ae554a16ca5665c9949f3f743386031d2ca7331d8ead81116 not found: ID does not exist"
Sep 09 16:36:18.817713 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:18.817606    2567 scope.go:117] "RemoveContainer" containerID="9789f34aadf1447209581745e02a80845903a2987068fa4d79b62ebc47434fbe"
Sep 09 16:36:18.818122 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:36:18.818101    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9789f34aadf1447209581745e02a80845903a2987068fa4d79b62ebc47434fbe\": container with ID starting with 9789f34aadf1447209581745e02a80845903a2987068fa4d79b62ebc47434fbe not found: ID does not exist" containerID="9789f34aadf1447209581745e02a80845903a2987068fa4d79b62ebc47434fbe"
Sep 09 16:36:18.818190 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:18.818128    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9789f34aadf1447209581745e02a80845903a2987068fa4d79b62ebc47434fbe"} err="failed to get container status \"9789f34aadf1447209581745e02a80845903a2987068fa4d79b62ebc47434fbe\": rpc error: code = NotFound desc = could not find container \"9789f34aadf1447209581745e02a80845903a2987068fa4d79b62ebc47434fbe\": container with ID starting with 9789f34aadf1447209581745e02a80845903a2987068fa4d79b62ebc47434fbe not found: ID does not exist"
Sep 09 16:36:18.819848 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:18.819827    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8"]
Sep 09 16:36:18.825632 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:18.825611    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-ab78ef-openshift-default-797cf4477b-h4jh8"]
Sep 09 16:36:19.036628 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:19.036537    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c864e699-e078-4530-bc10-da7da9e6cc0e" path="/var/lib/kubelet/pods/c864e699-e078-4530-bc10-da7da9e6cc0e/volumes"
Sep 09 16:36:19.520957 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:19.520920    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:36:28.733629 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:28.733592    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:36:31.254378 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:31.254346    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4"]
Sep 09 16:36:31.254781 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:31.254589    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4" podUID="de55f322-3dd3-4c96-967f-c99336cad8ac" containerName="istio-proxy" containerID="cri-o://04ef0ceb5b970451d72a976bc2b53827b2041bcc97498a2caadbce269b0e5f1c" gracePeriod=30
Sep 09 16:36:31.349851 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:31.349806    2567 patch_prober.go:28] interesting pod/e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 09 16:36:31.350047 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:31.349879    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4" podUID="de55f322-3dd3-4c96-967f-c99336cad8ac" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 16:36:36.511747 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:36.511721    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4"
Sep 09 16:36:36.639905 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:36.639866    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/de55f322-3dd3-4c96-967f-c99336cad8ac-workload-certs\") pod \"de55f322-3dd3-4c96-967f-c99336cad8ac\" (UID: \"de55f322-3dd3-4c96-967f-c99336cad8ac\") "
Sep 09 16:36:36.640110 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:36.639912    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q8tps\" (UniqueName: \"kubernetes.io/projected/de55f322-3dd3-4c96-967f-c99336cad8ac-kube-api-access-q8tps\") pod \"de55f322-3dd3-4c96-967f-c99336cad8ac\" (UID: \"de55f322-3dd3-4c96-967f-c99336cad8ac\") "
Sep 09 16:36:36.640110 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:36.639958    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/de55f322-3dd3-4c96-967f-c99336cad8ac-istio-data\") pod \"de55f322-3dd3-4c96-967f-c99336cad8ac\" (UID: \"de55f322-3dd3-4c96-967f-c99336cad8ac\") "
Sep 09 16:36:36.640110 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:36.640012    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/de55f322-3dd3-4c96-967f-c99336cad8ac-istio-envoy\") pod \"de55f322-3dd3-4c96-967f-c99336cad8ac\" (UID: \"de55f322-3dd3-4c96-967f-c99336cad8ac\") "
Sep 09 16:36:36.640271 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:36.640180    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/de55f322-3dd3-4c96-967f-c99336cad8ac-istio-podinfo\") pod \"de55f322-3dd3-4c96-967f-c99336cad8ac\" (UID: \"de55f322-3dd3-4c96-967f-c99336cad8ac\") "
Sep 09 16:36:36.640271 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:36.640199    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de55f322-3dd3-4c96-967f-c99336cad8ac-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "de55f322-3dd3-4c96-967f-c99336cad8ac" (UID: "de55f322-3dd3-4c96-967f-c99336cad8ac"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:36:36.640271 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:36.640219    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/de55f322-3dd3-4c96-967f-c99336cad8ac-credential-socket\") pod \"de55f322-3dd3-4c96-967f-c99336cad8ac\" (UID: \"de55f322-3dd3-4c96-967f-c99336cad8ac\") "
Sep 09 16:36:36.640271 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:36.640258    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/de55f322-3dd3-4c96-967f-c99336cad8ac-istiod-ca-cert\") pod \"de55f322-3dd3-4c96-967f-c99336cad8ac\" (UID: \"de55f322-3dd3-4c96-967f-c99336cad8ac\") "
Sep 09 16:36:36.640591 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:36.640268    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de55f322-3dd3-4c96-967f-c99336cad8ac-istio-data" (OuterVolumeSpecName: "istio-data") pod "de55f322-3dd3-4c96-967f-c99336cad8ac" (UID: "de55f322-3dd3-4c96-967f-c99336cad8ac"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:36:36.640591 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:36.640281    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/de55f322-3dd3-4c96-967f-c99336cad8ac-workload-socket\") pod \"de55f322-3dd3-4c96-967f-c99336cad8ac\" (UID: \"de55f322-3dd3-4c96-967f-c99336cad8ac\") "
Sep 09 16:36:36.640591 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:36.640360    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de55f322-3dd3-4c96-967f-c99336cad8ac-istio-token\") pod \"de55f322-3dd3-4c96-967f-c99336cad8ac\" (UID: \"de55f322-3dd3-4c96-967f-c99336cad8ac\") "
Sep 09 16:36:36.640591 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:36.640416    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de55f322-3dd3-4c96-967f-c99336cad8ac-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "de55f322-3dd3-4c96-967f-c99336cad8ac" (UID: "de55f322-3dd3-4c96-967f-c99336cad8ac"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:36:36.640969 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:36.640597    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de55f322-3dd3-4c96-967f-c99336cad8ac-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "de55f322-3dd3-4c96-967f-c99336cad8ac" (UID: "de55f322-3dd3-4c96-967f-c99336cad8ac"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:36:36.640969 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:36.640647    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de55f322-3dd3-4c96-967f-c99336cad8ac-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "de55f322-3dd3-4c96-967f-c99336cad8ac" (UID: "de55f322-3dd3-4c96-967f-c99336cad8ac"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 16:36:36.640969 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:36.640807    2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/de55f322-3dd3-4c96-967f-c99336cad8ac-istio-data\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:36:36.640969 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:36.640939    2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/de55f322-3dd3-4c96-967f-c99336cad8ac-credential-socket\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:36:36.640969 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:36.640955    2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/de55f322-3dd3-4c96-967f-c99336cad8ac-istiod-ca-cert\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:36:36.640969 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:36.640968    2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/de55f322-3dd3-4c96-967f-c99336cad8ac-workload-socket\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:36:36.641290 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:36.640981    2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/de55f322-3dd3-4c96-967f-c99336cad8ac-workload-certs\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:36:36.642495 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:36.642447    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/de55f322-3dd3-4c96-967f-c99336cad8ac-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "de55f322-3dd3-4c96-967f-c99336cad8ac" (UID: "de55f322-3dd3-4c96-967f-c99336cad8ac"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 16:36:36.642622 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:36.642565    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de55f322-3dd3-4c96-967f-c99336cad8ac-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "de55f322-3dd3-4c96-967f-c99336cad8ac" (UID: "de55f322-3dd3-4c96-967f-c99336cad8ac"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:36:36.642622 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:36.642596    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de55f322-3dd3-4c96-967f-c99336cad8ac-kube-api-access-q8tps" (OuterVolumeSpecName: "kube-api-access-q8tps") pod "de55f322-3dd3-4c96-967f-c99336cad8ac" (UID: "de55f322-3dd3-4c96-967f-c99336cad8ac"). InnerVolumeSpecName "kube-api-access-q8tps". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 16:36:36.642622 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:36.642600    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de55f322-3dd3-4c96-967f-c99336cad8ac-istio-token" (OuterVolumeSpecName: "istio-token") pod "de55f322-3dd3-4c96-967f-c99336cad8ac" (UID: "de55f322-3dd3-4c96-967f-c99336cad8ac"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 16:36:36.741966 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:36.741923    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q8tps\" (UniqueName: \"kubernetes.io/projected/de55f322-3dd3-4c96-967f-c99336cad8ac-kube-api-access-q8tps\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:36:36.741966 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:36.741960    2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/de55f322-3dd3-4c96-967f-c99336cad8ac-istio-envoy\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:36:36.741966 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:36.741973    2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/de55f322-3dd3-4c96-967f-c99336cad8ac-istio-podinfo\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:36:36.742210 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:36.741986    2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de55f322-3dd3-4c96-967f-c99336cad8ac-istio-token\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:36:36.889148 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:36.889116    2567 generic.go:356] "Generic (PLEG): container finished" podID="de55f322-3dd3-4c96-967f-c99336cad8ac" containerID="04ef0ceb5b970451d72a976bc2b53827b2041bcc97498a2caadbce269b0e5f1c" exitCode=0
Sep 09 16:36:36.889326 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:36.889167    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4"
Sep 09 16:36:36.889326 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:36.889171    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4" event={"ID":"de55f322-3dd3-4c96-967f-c99336cad8ac","Type":"ContainerDied","Data":"04ef0ceb5b970451d72a976bc2b53827b2041bcc97498a2caadbce269b0e5f1c"}
Sep 09 16:36:36.889326 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:36.889278    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4" event={"ID":"de55f322-3dd3-4c96-967f-c99336cad8ac","Type":"ContainerDied","Data":"afdfcc0f07249259a7a788d6f5cf9f84ccb412331a95f17f040b3a054645e792"}
Sep 09 16:36:36.889326 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:36.889305    2567 scope.go:117] "RemoveContainer" containerID="04ef0ceb5b970451d72a976bc2b53827b2041bcc97498a2caadbce269b0e5f1c"
Sep 09 16:36:36.905293 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:36.905271    2567 scope.go:117] "RemoveContainer" containerID="8e02c81b37ff3bb3fe0e7a2ab9b2887e90bfd7225a0c892ce632d137e8d09099"
Sep 09 16:36:36.916532 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:36.916508    2567 scope.go:117] "RemoveContainer" containerID="04ef0ceb5b970451d72a976bc2b53827b2041bcc97498a2caadbce269b0e5f1c"
Sep 09 16:36:36.916610 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:36.916529    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4"]
Sep 09 16:36:36.916821 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:36:36.916803    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04ef0ceb5b970451d72a976bc2b53827b2041bcc97498a2caadbce269b0e5f1c\": container with ID starting with 04ef0ceb5b970451d72a976bc2b53827b2041bcc97498a2caadbce269b0e5f1c not found: ID does not exist" containerID="04ef0ceb5b970451d72a976bc2b53827b2041bcc97498a2caadbce269b0e5f1c"
Sep 09 16:36:36.916861 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:36.916831    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04ef0ceb5b970451d72a976bc2b53827b2041bcc97498a2caadbce269b0e5f1c"} err="failed to get container status \"04ef0ceb5b970451d72a976bc2b53827b2041bcc97498a2caadbce269b0e5f1c\": rpc error: code = NotFound desc = could not find container \"04ef0ceb5b970451d72a976bc2b53827b2041bcc97498a2caadbce269b0e5f1c\": container with ID starting with 04ef0ceb5b970451d72a976bc2b53827b2041bcc97498a2caadbce269b0e5f1c not found: ID does not exist"
Sep 09 16:36:36.916861 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:36.916853    2567 scope.go:117] "RemoveContainer" containerID="8e02c81b37ff3bb3fe0e7a2ab9b2887e90bfd7225a0c892ce632d137e8d09099"
Sep 09 16:36:36.917160 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:36:36.917140    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e02c81b37ff3bb3fe0e7a2ab9b2887e90bfd7225a0c892ce632d137e8d09099\": container with ID starting with 8e02c81b37ff3bb3fe0e7a2ab9b2887e90bfd7225a0c892ce632d137e8d09099 not found: ID does not exist" containerID="8e02c81b37ff3bb3fe0e7a2ab9b2887e90bfd7225a0c892ce632d137e8d09099"
Sep 09 16:36:36.917253 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:36.917170    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e02c81b37ff3bb3fe0e7a2ab9b2887e90bfd7225a0c892ce632d137e8d09099"} err="failed to get container status \"8e02c81b37ff3bb3fe0e7a2ab9b2887e90bfd7225a0c892ce632d137e8d09099\": rpc error: code = NotFound desc = could not find container \"8e02c81b37ff3bb3fe0e7a2ab9b2887e90bfd7225a0c892ce632d137e8d09099\": container with ID starting with 8e02c81b37ff3bb3fe0e7a2ab9b2887e90bfd7225a0c892ce632d137e8d09099 not found: ID does not exist"
Sep 09 16:36:36.920399 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:36.920378    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-042f8b-openshift-default-6c9fb6965d-ggkb4"]
Sep 09 16:36:37.036517 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:37.036483    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de55f322-3dd3-4c96-967f-c99336cad8ac" path="/var/lib/kubelet/pods/de55f322-3dd3-4c96-967f-c99336cad8ac/volumes"
Sep 09 16:36:42.048149 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:42.048113    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-de97ae-558ddff7f7-zh2h7"]
Sep 09 16:36:42.048673 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:42.048339    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-de97ae-558ddff7f7-zh2h7" podUID="6a2a3589-419e-47ba-bc4a-bbc4ddc9ff3e" containerName="maas-api" containerID="cri-o://8f744da3711135bea583f4d367e067fef993b15b8dd5fcc3a50a2f7383568303" gracePeriod=30
Sep 09 16:36:42.297020 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:42.296993    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-de97ae-558ddff7f7-zh2h7"
Sep 09 16:36:42.398416 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:42.398380    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xgprf\" (UniqueName: \"kubernetes.io/projected/6a2a3589-419e-47ba-bc4a-bbc4ddc9ff3e-kube-api-access-xgprf\") pod \"6a2a3589-419e-47ba-bc4a-bbc4ddc9ff3e\" (UID: \"6a2a3589-419e-47ba-bc4a-bbc4ddc9ff3e\") "
Sep 09 16:36:42.398416 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:42.398423    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6a2a3589-419e-47ba-bc4a-bbc4ddc9ff3e-maas-api-tls\") pod \"6a2a3589-419e-47ba-bc4a-bbc4ddc9ff3e\" (UID: \"6a2a3589-419e-47ba-bc4a-bbc4ddc9ff3e\") "
Sep 09 16:36:42.398726 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:42.398532    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a2a3589-419e-47ba-bc4a-bbc4ddc9ff3e-openshift-service-ca\") pod \"6a2a3589-419e-47ba-bc4a-bbc4ddc9ff3e\" (UID: \"6a2a3589-419e-47ba-bc4a-bbc4ddc9ff3e\") "
Sep 09 16:36:42.398726 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:42.398602    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6a2a3589-419e-47ba-bc4a-bbc4ddc9ff3e-metrics-certs\") pod \"6a2a3589-419e-47ba-bc4a-bbc4ddc9ff3e\" (UID: \"6a2a3589-419e-47ba-bc4a-bbc4ddc9ff3e\") "
Sep 09 16:36:42.398944 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:42.398920    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a2a3589-419e-47ba-bc4a-bbc4ddc9ff3e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6a2a3589-419e-47ba-bc4a-bbc4ddc9ff3e" (UID: "6a2a3589-419e-47ba-bc4a-bbc4ddc9ff3e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 16:36:42.400860 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:42.400836    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a2a3589-419e-47ba-bc4a-bbc4ddc9ff3e-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "6a2a3589-419e-47ba-bc4a-bbc4ddc9ff3e" (UID: "6a2a3589-419e-47ba-bc4a-bbc4ddc9ff3e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 16:36:42.400954 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:42.400857    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a2a3589-419e-47ba-bc4a-bbc4ddc9ff3e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6a2a3589-419e-47ba-bc4a-bbc4ddc9ff3e" (UID: "6a2a3589-419e-47ba-bc4a-bbc4ddc9ff3e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 16:36:42.400954 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:42.400872    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a2a3589-419e-47ba-bc4a-bbc4ddc9ff3e-kube-api-access-xgprf" (OuterVolumeSpecName: "kube-api-access-xgprf") pod "6a2a3589-419e-47ba-bc4a-bbc4ddc9ff3e" (UID: "6a2a3589-419e-47ba-bc4a-bbc4ddc9ff3e"). InnerVolumeSpecName "kube-api-access-xgprf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 16:36:42.499605 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:42.499573    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a2a3589-419e-47ba-bc4a-bbc4ddc9ff3e-openshift-service-ca\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:36:42.499605 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:42.499601    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6a2a3589-419e-47ba-bc4a-bbc4ddc9ff3e-metrics-certs\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:36:42.499605 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:42.499610    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xgprf\" (UniqueName: \"kubernetes.io/projected/6a2a3589-419e-47ba-bc4a-bbc4ddc9ff3e-kube-api-access-xgprf\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:36:42.499842 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:42.499620    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6a2a3589-419e-47ba-bc4a-bbc4ddc9ff3e-maas-api-tls\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:36:42.710170 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:42.710074    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:36:42.925552 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:42.925511    2567 generic.go:356] "Generic (PLEG): container finished" podID="6a2a3589-419e-47ba-bc4a-bbc4ddc9ff3e" containerID="8f744da3711135bea583f4d367e067fef993b15b8dd5fcc3a50a2f7383568303" exitCode=0
Sep 09 16:36:42.925724 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:42.925574    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-de97ae-558ddff7f7-zh2h7"
Sep 09 16:36:42.925724 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:42.925600    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-de97ae-558ddff7f7-zh2h7" event={"ID":"6a2a3589-419e-47ba-bc4a-bbc4ddc9ff3e","Type":"ContainerDied","Data":"8f744da3711135bea583f4d367e067fef993b15b8dd5fcc3a50a2f7383568303"}
Sep 09 16:36:42.925724 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:42.925676    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-de97ae-558ddff7f7-zh2h7" event={"ID":"6a2a3589-419e-47ba-bc4a-bbc4ddc9ff3e","Type":"ContainerDied","Data":"4405dfcfc9de9ed9d151da7eff3f6dbf7a9126c4e690b0f8e7738fa78527ddbd"}
Sep 09 16:36:42.925724 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:42.925699    2567 scope.go:117] "RemoveContainer" containerID="8f744da3711135bea583f4d367e067fef993b15b8dd5fcc3a50a2f7383568303"
Sep 09 16:36:42.935006 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:42.934984    2567 scope.go:117] "RemoveContainer" containerID="8f744da3711135bea583f4d367e067fef993b15b8dd5fcc3a50a2f7383568303"
Sep 09 16:36:42.935280 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:36:42.935262    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f744da3711135bea583f4d367e067fef993b15b8dd5fcc3a50a2f7383568303\": container with ID starting with 8f744da3711135bea583f4d367e067fef993b15b8dd5fcc3a50a2f7383568303 not found: ID does not exist" containerID="8f744da3711135bea583f4d367e067fef993b15b8dd5fcc3a50a2f7383568303"
Sep 09 16:36:42.935346 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:42.935286    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f744da3711135bea583f4d367e067fef993b15b8dd5fcc3a50a2f7383568303"} err="failed to get container status \"8f744da3711135bea583f4d367e067fef993b15b8dd5fcc3a50a2f7383568303\": rpc error: code = NotFound desc = could not find container \"8f744da3711135bea583f4d367e067fef993b15b8dd5fcc3a50a2f7383568303\": container with ID starting with 8f744da3711135bea583f4d367e067fef993b15b8dd5fcc3a50a2f7383568303 not found: ID does not exist"
Sep 09 16:36:42.947691 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:42.947668    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-de97ae-558ddff7f7-zh2h7"]
Sep 09 16:36:42.951099 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:42.951073    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-de97ae-558ddff7f7-zh2h7"]
Sep 09 16:36:43.036137 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:43.036052    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a2a3589-419e-47ba-bc4a-bbc4ddc9ff3e" path="/var/lib/kubelet/pods/6a2a3589-419e-47ba-bc4a-bbc4ddc9ff3e/volumes"
Sep 09 16:36:48.606168 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:48.606128    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m"]
Sep 09 16:36:48.606672 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:48.606437    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m" podUID="5c55fb39-101d-4ba1-9f9d-ff6a97232826" containerName="istio-proxy" containerID="cri-o://9aa6ce252c047ad10ff0d02759c68d2bfaac84ae6ab277970644f5ffe633374c" gracePeriod=30
Sep 09 16:36:53.867956 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:53.867932    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m"
Sep 09 16:36:53.908110 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:53.908037    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c55fb39-101d-4ba1-9f9d-ff6a97232826-istio-data\") pod \"5c55fb39-101d-4ba1-9f9d-ff6a97232826\" (UID: \"5c55fb39-101d-4ba1-9f9d-ff6a97232826\") "
Sep 09 16:36:53.908110 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:53.908085    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c55fb39-101d-4ba1-9f9d-ff6a97232826-istiod-ca-cert\") pod \"5c55fb39-101d-4ba1-9f9d-ff6a97232826\" (UID: \"5c55fb39-101d-4ba1-9f9d-ff6a97232826\") "
Sep 09 16:36:53.908298 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:53.908124    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c55fb39-101d-4ba1-9f9d-ff6a97232826-istio-envoy\") pod \"5c55fb39-101d-4ba1-9f9d-ff6a97232826\" (UID: \"5c55fb39-101d-4ba1-9f9d-ff6a97232826\") "
Sep 09 16:36:53.908298 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:53.908224    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c55fb39-101d-4ba1-9f9d-ff6a97232826-istio-token\") pod \"5c55fb39-101d-4ba1-9f9d-ff6a97232826\" (UID: \"5c55fb39-101d-4ba1-9f9d-ff6a97232826\") "
Sep 09 16:36:53.908298 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:53.908252    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c55fb39-101d-4ba1-9f9d-ff6a97232826-credential-socket\") pod \"5c55fb39-101d-4ba1-9f9d-ff6a97232826\" (UID: \"5c55fb39-101d-4ba1-9f9d-ff6a97232826\") "
Sep 09 16:36:53.908298 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:53.908295    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c55fb39-101d-4ba1-9f9d-ff6a97232826-workload-socket\") pod \"5c55fb39-101d-4ba1-9f9d-ff6a97232826\" (UID: \"5c55fb39-101d-4ba1-9f9d-ff6a97232826\") "
Sep 09 16:36:53.908846 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:53.908318    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c55fb39-101d-4ba1-9f9d-ff6a97232826-workload-certs\") pod \"5c55fb39-101d-4ba1-9f9d-ff6a97232826\" (UID: \"5c55fb39-101d-4ba1-9f9d-ff6a97232826\") "
Sep 09 16:36:53.908846 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:53.908341    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c55fb39-101d-4ba1-9f9d-ff6a97232826-istio-data" (OuterVolumeSpecName: "istio-data") pod "5c55fb39-101d-4ba1-9f9d-ff6a97232826" (UID: "5c55fb39-101d-4ba1-9f9d-ff6a97232826"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:36:53.908846 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:53.908357    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-phfdt\" (UniqueName: \"kubernetes.io/projected/5c55fb39-101d-4ba1-9f9d-ff6a97232826-kube-api-access-phfdt\") pod \"5c55fb39-101d-4ba1-9f9d-ff6a97232826\" (UID: \"5c55fb39-101d-4ba1-9f9d-ff6a97232826\") "
Sep 09 16:36:53.908846 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:53.908384    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c55fb39-101d-4ba1-9f9d-ff6a97232826-istio-podinfo\") pod \"5c55fb39-101d-4ba1-9f9d-ff6a97232826\" (UID: \"5c55fb39-101d-4ba1-9f9d-ff6a97232826\") "
Sep 09 16:36:53.908846 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:53.908562    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c55fb39-101d-4ba1-9f9d-ff6a97232826-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5c55fb39-101d-4ba1-9f9d-ff6a97232826" (UID: "5c55fb39-101d-4ba1-9f9d-ff6a97232826"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:36:53.908846 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:53.908580    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c55fb39-101d-4ba1-9f9d-ff6a97232826-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5c55fb39-101d-4ba1-9f9d-ff6a97232826" (UID: "5c55fb39-101d-4ba1-9f9d-ff6a97232826"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:36:53.908846 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:53.908704    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c55fb39-101d-4ba1-9f9d-ff6a97232826-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5c55fb39-101d-4ba1-9f9d-ff6a97232826" (UID: "5c55fb39-101d-4ba1-9f9d-ff6a97232826"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 16:36:53.908846 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:53.908740    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c55fb39-101d-4ba1-9f9d-ff6a97232826-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5c55fb39-101d-4ba1-9f9d-ff6a97232826" (UID: "5c55fb39-101d-4ba1-9f9d-ff6a97232826"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:36:53.908846 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:53.908801    2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c55fb39-101d-4ba1-9f9d-ff6a97232826-istiod-ca-cert\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:36:53.908846 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:53.908821    2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c55fb39-101d-4ba1-9f9d-ff6a97232826-credential-socket\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:36:53.908846 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:53.908835    2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c55fb39-101d-4ba1-9f9d-ff6a97232826-workload-socket\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:36:53.908846 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:53.908848    2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c55fb39-101d-4ba1-9f9d-ff6a97232826-istio-data\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:36:53.910710 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:53.910686    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c55fb39-101d-4ba1-9f9d-ff6a97232826-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5c55fb39-101d-4ba1-9f9d-ff6a97232826" (UID: "5c55fb39-101d-4ba1-9f9d-ff6a97232826"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:36:53.910792 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:53.910766    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c55fb39-101d-4ba1-9f9d-ff6a97232826-istio-token" (OuterVolumeSpecName: "istio-token") pod "5c55fb39-101d-4ba1-9f9d-ff6a97232826" (UID: "5c55fb39-101d-4ba1-9f9d-ff6a97232826"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 16:36:53.911223 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:53.911202    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5c55fb39-101d-4ba1-9f9d-ff6a97232826-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5c55fb39-101d-4ba1-9f9d-ff6a97232826" (UID: "5c55fb39-101d-4ba1-9f9d-ff6a97232826"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 16:36:53.911272 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:53.911212    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c55fb39-101d-4ba1-9f9d-ff6a97232826-kube-api-access-phfdt" (OuterVolumeSpecName: "kube-api-access-phfdt") pod "5c55fb39-101d-4ba1-9f9d-ff6a97232826" (UID: "5c55fb39-101d-4ba1-9f9d-ff6a97232826"). InnerVolumeSpecName "kube-api-access-phfdt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 16:36:53.980429 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:53.980393    2567 generic.go:356] "Generic (PLEG): container finished" podID="5c55fb39-101d-4ba1-9f9d-ff6a97232826" containerID="9aa6ce252c047ad10ff0d02759c68d2bfaac84ae6ab277970644f5ffe633374c" exitCode=0
Sep 09 16:36:53.980606 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:53.980445    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m"
Sep 09 16:36:53.980606 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:53.980485    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m" event={"ID":"5c55fb39-101d-4ba1-9f9d-ff6a97232826","Type":"ContainerDied","Data":"9aa6ce252c047ad10ff0d02759c68d2bfaac84ae6ab277970644f5ffe633374c"}
Sep 09 16:36:53.980606 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:53.980533    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m" event={"ID":"5c55fb39-101d-4ba1-9f9d-ff6a97232826","Type":"ContainerDied","Data":"4cd4d7a6cc12d92c20042a4af0d05177d0afee2301a03a7f91c30965ba997d29"}
Sep 09 16:36:53.980606 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:53.980554    2567 scope.go:117] "RemoveContainer" containerID="9aa6ce252c047ad10ff0d02759c68d2bfaac84ae6ab277970644f5ffe633374c"
Sep 09 16:36:53.989966 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:53.989948    2567 scope.go:117] "RemoveContainer" containerID="1948df54257d60064fa43c7205e1afaa325b400477ed0c89a204244d06ed37c1"
Sep 09 16:36:53.998007 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:53.997986    2567 scope.go:117] "RemoveContainer" containerID="9aa6ce252c047ad10ff0d02759c68d2bfaac84ae6ab277970644f5ffe633374c"
Sep 09 16:36:53.998283 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:36:53.998263    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9aa6ce252c047ad10ff0d02759c68d2bfaac84ae6ab277970644f5ffe633374c\": container with ID starting with 9aa6ce252c047ad10ff0d02759c68d2bfaac84ae6ab277970644f5ffe633374c not found: ID does not exist" containerID="9aa6ce252c047ad10ff0d02759c68d2bfaac84ae6ab277970644f5ffe633374c"
Sep 09 16:36:53.998335 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:53.998293    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9aa6ce252c047ad10ff0d02759c68d2bfaac84ae6ab277970644f5ffe633374c"} err="failed to get container status \"9aa6ce252c047ad10ff0d02759c68d2bfaac84ae6ab277970644f5ffe633374c\": rpc error: code = NotFound desc = could not find container \"9aa6ce252c047ad10ff0d02759c68d2bfaac84ae6ab277970644f5ffe633374c\": container with ID starting with 9aa6ce252c047ad10ff0d02759c68d2bfaac84ae6ab277970644f5ffe633374c not found: ID does not exist"
Sep 09 16:36:53.998335 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:53.998310    2567 scope.go:117] "RemoveContainer" containerID="1948df54257d60064fa43c7205e1afaa325b400477ed0c89a204244d06ed37c1"
Sep 09 16:36:53.998650 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:36:53.998632    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1948df54257d60064fa43c7205e1afaa325b400477ed0c89a204244d06ed37c1\": container with ID starting with 1948df54257d60064fa43c7205e1afaa325b400477ed0c89a204244d06ed37c1 not found: ID does not exist" containerID="1948df54257d60064fa43c7205e1afaa325b400477ed0c89a204244d06ed37c1"
Sep 09 16:36:53.998719 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:53.998653    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1948df54257d60064fa43c7205e1afaa325b400477ed0c89a204244d06ed37c1"} err="failed to get container status \"1948df54257d60064fa43c7205e1afaa325b400477ed0c89a204244d06ed37c1\": rpc error: code = NotFound desc = could not find container \"1948df54257d60064fa43c7205e1afaa325b400477ed0c89a204244d06ed37c1\": container with ID starting with 1948df54257d60064fa43c7205e1afaa325b400477ed0c89a204244d06ed37c1 not found: ID does not exist"
Sep 09 16:36:54.001513 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:54.001492    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m"]
Sep 09 16:36:54.003401 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:54.003384    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-de97ae-openshift-default-698b59f4c4-5gf6m"]
Sep 09 16:36:54.010022 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:54.010003    2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c55fb39-101d-4ba1-9f9d-ff6a97232826-istio-token\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:36:54.010022 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:54.010024    2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c55fb39-101d-4ba1-9f9d-ff6a97232826-workload-certs\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:36:54.010147 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:54.010033    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-phfdt\" (UniqueName: \"kubernetes.io/projected/5c55fb39-101d-4ba1-9f9d-ff6a97232826-kube-api-access-phfdt\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:36:54.010147 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:54.010041    2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c55fb39-101d-4ba1-9f9d-ff6a97232826-istio-podinfo\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:36:54.010147 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:54.010050    2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c55fb39-101d-4ba1-9f9d-ff6a97232826-istio-envoy\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:36:55.035681 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:55.035646    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c55fb39-101d-4ba1-9f9d-ff6a97232826" path="/var/lib/kubelet/pods/5c55fb39-101d-4ba1-9f9d-ff6a97232826/volumes"
Sep 09 16:36:55.709254 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:36:55.709220    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:37:04.493875 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:04.493841    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp"]
Sep 09 16:37:04.494336 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:04.494216    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8dedd14-b232-4636-9f89-8552b16c6d94" containerName="istio-proxy"
Sep 09 16:37:04.494336 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:04.494230    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8dedd14-b232-4636-9f89-8552b16c6d94" containerName="istio-proxy"
Sep 09 16:37:04.494336 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:04.494244    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c55fb39-101d-4ba1-9f9d-ff6a97232826" containerName="istio-proxy"
Sep 09 16:37:04.494336 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:04.494253    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c55fb39-101d-4ba1-9f9d-ff6a97232826" containerName="istio-proxy"
Sep 09 16:37:04.494336 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:04.494259    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50995643-35be-45c9-b756-280a7d978dc3" containerName="maas-api"
Sep 09 16:37:04.494336 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:04.494265    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="50995643-35be-45c9-b756-280a7d978dc3" containerName="maas-api"
Sep 09 16:37:04.494336 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:04.494273    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de55f322-3dd3-4c96-967f-c99336cad8ac" containerName="istio-proxy"
Sep 09 16:37:04.494336 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:04.494279    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="de55f322-3dd3-4c96-967f-c99336cad8ac" containerName="istio-proxy"
Sep 09 16:37:04.494336 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:04.494284    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c864e699-e078-4530-bc10-da7da9e6cc0e" containerName="istio-proxy"
Sep 09 16:37:04.494336 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:04.494289    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c864e699-e078-4530-bc10-da7da9e6cc0e" containerName="istio-proxy"
Sep 09 16:37:04.494336 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:04.494305    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a2a3589-419e-47ba-bc4a-bbc4ddc9ff3e" containerName="maas-api"
Sep 09 16:37:04.494336 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:04.494310    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a2a3589-419e-47ba-bc4a-bbc4ddc9ff3e" containerName="maas-api"
Sep 09 16:37:04.494336 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:04.494322    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de55f322-3dd3-4c96-967f-c99336cad8ac" containerName="istio-proxy"
Sep 09 16:37:04.494336 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:04.494330    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="de55f322-3dd3-4c96-967f-c99336cad8ac" containerName="istio-proxy"
Sep 09 16:37:04.494336 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:04.494336    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c864e699-e078-4530-bc10-da7da9e6cc0e" containerName="istio-proxy"
Sep 09 16:37:04.494336 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:04.494341    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c864e699-e078-4530-bc10-da7da9e6cc0e" containerName="istio-proxy"
Sep 09 16:37:04.494877 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:04.494404    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8dedd14-b232-4636-9f89-8552b16c6d94" containerName="istio-proxy"
Sep 09 16:37:04.494877 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:04.494413    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a2a3589-419e-47ba-bc4a-bbc4ddc9ff3e" containerName="maas-api"
Sep 09 16:37:04.494877 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:04.494420    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="c864e699-e078-4530-bc10-da7da9e6cc0e" containerName="istio-proxy"
Sep 09 16:37:04.494877 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:04.494425    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="de55f322-3dd3-4c96-967f-c99336cad8ac" containerName="istio-proxy"
Sep 09 16:37:04.494877 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:04.494431    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="de55f322-3dd3-4c96-967f-c99336cad8ac" containerName="istio-proxy"
Sep 09 16:37:04.494877 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:04.494436    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c55fb39-101d-4ba1-9f9d-ff6a97232826" containerName="istio-proxy"
Sep 09 16:37:04.494877 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:04.494442    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="c864e699-e078-4530-bc10-da7da9e6cc0e" containerName="istio-proxy"
Sep 09 16:37:04.494877 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:04.494447    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="50995643-35be-45c9-b756-280a7d978dc3" containerName="maas-api"
Sep 09 16:37:04.499102 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:04.499082    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp"
Sep 09 16:37:04.501667 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:04.501645    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-54370d-openshift-default-dockercfg-chsqw\""
Sep 09 16:37:04.508357 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:04.508325    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp"]
Sep 09 16:37:04.610600 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:04.610557    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e5b5a13-128c-4713-bd42-fb9de49b5838-istio-token\") pod \"e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp\" (UID: \"4e5b5a13-128c-4713-bd42-fb9de49b5838\") " pod="openshift-ingress/e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp"
Sep 09 16:37:04.610600 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:04.610600    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e5b5a13-128c-4713-bd42-fb9de49b5838-credential-socket\") pod \"e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp\" (UID: \"4e5b5a13-128c-4713-bd42-fb9de49b5838\") " pod="openshift-ingress/e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp"
Sep 09 16:37:04.610841 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:04.610621    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e5b5a13-128c-4713-bd42-fb9de49b5838-istio-envoy\") pod \"e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp\" (UID: \"4e5b5a13-128c-4713-bd42-fb9de49b5838\") " pod="openshift-ingress/e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp"
Sep 09 16:37:04.610841 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:04.610649    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e5b5a13-128c-4713-bd42-fb9de49b5838-istio-podinfo\") pod \"e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp\" (UID: \"4e5b5a13-128c-4713-bd42-fb9de49b5838\") " pod="openshift-ingress/e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp"
Sep 09 16:37:04.610841 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:04.610737    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e5b5a13-128c-4713-bd42-fb9de49b5838-workload-certs\") pod \"e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp\" (UID: \"4e5b5a13-128c-4713-bd42-fb9de49b5838\") " pod="openshift-ingress/e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp"
Sep 09 16:37:04.610841 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:04.610800    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e5b5a13-128c-4713-bd42-fb9de49b5838-istio-data\") pod \"e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp\" (UID: \"4e5b5a13-128c-4713-bd42-fb9de49b5838\") " pod="openshift-ingress/e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp"
Sep 09 16:37:04.610841 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:04.610822    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9khtq\" (UniqueName: \"kubernetes.io/projected/4e5b5a13-128c-4713-bd42-fb9de49b5838-kube-api-access-9khtq\") pod \"e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp\" (UID: \"4e5b5a13-128c-4713-bd42-fb9de49b5838\") " pod="openshift-ingress/e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp"
Sep 09 16:37:04.611055 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:04.610867    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e5b5a13-128c-4713-bd42-fb9de49b5838-istiod-ca-cert\") pod \"e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp\" (UID: \"4e5b5a13-128c-4713-bd42-fb9de49b5838\") " pod="openshift-ingress/e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp"
Sep 09 16:37:04.611055 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:04.610979    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e5b5a13-128c-4713-bd42-fb9de49b5838-workload-socket\") pod \"e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp\" (UID: \"4e5b5a13-128c-4713-bd42-fb9de49b5838\") " pod="openshift-ingress/e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp"
Sep 09 16:37:04.712203 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:04.712162    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e5b5a13-128c-4713-bd42-fb9de49b5838-istio-token\") pod \"e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp\" (UID: \"4e5b5a13-128c-4713-bd42-fb9de49b5838\") " pod="openshift-ingress/e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp"
Sep 09 16:37:04.712203 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:04.712206    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e5b5a13-128c-4713-bd42-fb9de49b5838-credential-socket\") pod \"e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp\" (UID: \"4e5b5a13-128c-4713-bd42-fb9de49b5838\") " pod="openshift-ingress/e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp"
Sep 09 16:37:04.712433 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:04.712225    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e5b5a13-128c-4713-bd42-fb9de49b5838-istio-envoy\") pod \"e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp\" (UID: \"4e5b5a13-128c-4713-bd42-fb9de49b5838\") " pod="openshift-ingress/e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp"
Sep 09 16:37:04.712433 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:04.712251    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e5b5a13-128c-4713-bd42-fb9de49b5838-istio-podinfo\") pod \"e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp\" (UID: \"4e5b5a13-128c-4713-bd42-fb9de49b5838\") " pod="openshift-ingress/e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp"
Sep 09 16:37:04.712433 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:04.712295    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e5b5a13-128c-4713-bd42-fb9de49b5838-workload-certs\") pod \"e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp\" (UID: \"4e5b5a13-128c-4713-bd42-fb9de49b5838\") " pod="openshift-ingress/e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp"
Sep 09 16:37:04.712433 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:04.712351    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e5b5a13-128c-4713-bd42-fb9de49b5838-istio-data\") pod \"e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp\" (UID: \"4e5b5a13-128c-4713-bd42-fb9de49b5838\") " pod="openshift-ingress/e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp"
Sep 09 16:37:04.712668 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:04.712501    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9khtq\" (UniqueName: \"kubernetes.io/projected/4e5b5a13-128c-4713-bd42-fb9de49b5838-kube-api-access-9khtq\") pod \"e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp\" (UID: \"4e5b5a13-128c-4713-bd42-fb9de49b5838\") " pod="openshift-ingress/e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp"
Sep 09 16:37:04.712668 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:04.712586    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e5b5a13-128c-4713-bd42-fb9de49b5838-istiod-ca-cert\") pod \"e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp\" (UID: \"4e5b5a13-128c-4713-bd42-fb9de49b5838\") " pod="openshift-ingress/e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp"
Sep 09 16:37:04.712668 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:04.712641    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e5b5a13-128c-4713-bd42-fb9de49b5838-workload-socket\") pod \"e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp\" (UID: \"4e5b5a13-128c-4713-bd42-fb9de49b5838\") " pod="openshift-ingress/e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp"
Sep 09 16:37:04.712815 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:04.712739    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e5b5a13-128c-4713-bd42-fb9de49b5838-credential-socket\") pod \"e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp\" (UID: \"4e5b5a13-128c-4713-bd42-fb9de49b5838\") " pod="openshift-ingress/e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp"
Sep 09 16:37:04.712815 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:04.712804    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e5b5a13-128c-4713-bd42-fb9de49b5838-workload-certs\") pod \"e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp\" (UID: \"4e5b5a13-128c-4713-bd42-fb9de49b5838\") " pod="openshift-ingress/e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp"
Sep 09 16:37:04.712929 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:04.712739    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e5b5a13-128c-4713-bd42-fb9de49b5838-istio-data\") pod \"e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp\" (UID: \"4e5b5a13-128c-4713-bd42-fb9de49b5838\") " pod="openshift-ingress/e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp"
Sep 09 16:37:04.713012 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:04.712984    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e5b5a13-128c-4713-bd42-fb9de49b5838-workload-socket\") pod \"e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp\" (UID: \"4e5b5a13-128c-4713-bd42-fb9de49b5838\") " pod="openshift-ingress/e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp"
Sep 09 16:37:04.713383 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:04.713361    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e5b5a13-128c-4713-bd42-fb9de49b5838-istiod-ca-cert\") pod \"e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp\" (UID: \"4e5b5a13-128c-4713-bd42-fb9de49b5838\") " pod="openshift-ingress/e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp"
Sep 09 16:37:04.715192 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:04.715170    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e5b5a13-128c-4713-bd42-fb9de49b5838-istio-envoy\") pod \"e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp\" (UID: \"4e5b5a13-128c-4713-bd42-fb9de49b5838\") " pod="openshift-ingress/e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp"
Sep 09 16:37:04.715388 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:04.715370    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e5b5a13-128c-4713-bd42-fb9de49b5838-istio-podinfo\") pod \"e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp\" (UID: \"4e5b5a13-128c-4713-bd42-fb9de49b5838\") " pod="openshift-ingress/e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp"
Sep 09 16:37:04.723926 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:04.723902    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e5b5a13-128c-4713-bd42-fb9de49b5838-istio-token\") pod \"e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp\" (UID: \"4e5b5a13-128c-4713-bd42-fb9de49b5838\") " pod="openshift-ingress/e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp"
Sep 09 16:37:04.724142 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:04.724123    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9khtq\" (UniqueName: \"kubernetes.io/projected/4e5b5a13-128c-4713-bd42-fb9de49b5838-kube-api-access-9khtq\") pod \"e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp\" (UID: \"4e5b5a13-128c-4713-bd42-fb9de49b5838\") " pod="openshift-ingress/e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp"
Sep 09 16:37:04.810881 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:04.810791    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp"
Sep 09 16:37:05.090706 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:05.090680    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp"]
Sep 09 16:37:05.092547 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:37:05.092523    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e5b5a13_128c_4713_bd42_fb9de49b5838.slice/crio-f4567842d6b8d636cc89df002961ef50cc1635d77b93a00ae41e7ee3d62e99aa WatchSource:0}: Error finding container f4567842d6b8d636cc89df002961ef50cc1635d77b93a00ae41e7ee3d62e99aa: Status 404 returned error can't find the container with id f4567842d6b8d636cc89df002961ef50cc1635d77b93a00ae41e7ee3d62e99aa
Sep 09 16:37:05.094814 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:05.094777    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:37:05.094923 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:05.094846    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:37:05.094923 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:05.094879    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:37:06.037847 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:06.037766    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp" event={"ID":"4e5b5a13-128c-4713-bd42-fb9de49b5838","Type":"ContainerStarted","Data":"3ce26658914895da58fcd28521aad7b68fa1b079d73f7ecf995aced2d136a49f"}
Sep 09 16:37:06.037847 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:06.037831    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp" event={"ID":"4e5b5a13-128c-4713-bd42-fb9de49b5838","Type":"ContainerStarted","Data":"f4567842d6b8d636cc89df002961ef50cc1635d77b93a00ae41e7ee3d62e99aa"}
Sep 09 16:37:06.073094 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:06.073037    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp" podStartSLOduration=2.073022061 podStartE2EDuration="2.073022061s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 16:37:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 16:37:06.070284933 +0000 UTC m=+1701.488134932" watchObservedRunningTime="2026-09-09 16:37:06.073022061 +0000 UTC m=+1701.490872059"
Sep 09 16:37:06.631708 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:06.631671    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:37:06.811548 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:06.811507    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp"
Sep 09 16:37:06.816829 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:06.816799    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp"
Sep 09 16:37:07.042395 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:07.042315    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp"
Sep 09 16:37:07.043424 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:07.043403    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp"
Sep 09 16:37:09.227355 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:09.227320    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-54370d-84b4f44586-66tqw"]
Sep 09 16:37:09.228658 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:09.228628    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c55fb39-101d-4ba1-9f9d-ff6a97232826" containerName="istio-proxy"
Sep 09 16:37:09.228658 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:09.228654    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c55fb39-101d-4ba1-9f9d-ff6a97232826" containerName="istio-proxy"
Sep 09 16:37:09.228814 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:09.228701    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8dedd14-b232-4636-9f89-8552b16c6d94" containerName="istio-proxy"
Sep 09 16:37:09.228814 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:09.228709    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8dedd14-b232-4636-9f89-8552b16c6d94" containerName="istio-proxy"
Sep 09 16:37:09.228814 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:09.228794    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8dedd14-b232-4636-9f89-8552b16c6d94" containerName="istio-proxy"
Sep 09 16:37:09.228814 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:09.228807    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c55fb39-101d-4ba1-9f9d-ff6a97232826" containerName="istio-proxy"
Sep 09 16:37:09.232376 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:09.232358    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-54370d-84b4f44586-66tqw"
Sep 09 16:37:09.234850 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:09.234825    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-54370d-serving-cert\""
Sep 09 16:37:09.241884 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:09.241331    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-54370d-84b4f44586-66tqw"]
Sep 09 16:37:09.355825 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:09.355787    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djk2w\" (UniqueName: \"kubernetes.io/projected/447f2a14-e79a-4ffa-b257-0eb0d7cf82c7-kube-api-access-djk2w\") pod \"maas-api-e2e-ipp-a-54370d-84b4f44586-66tqw\" (UID: \"447f2a14-e79a-4ffa-b257-0eb0d7cf82c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-54370d-84b4f44586-66tqw"
Sep 09 16:37:09.356011 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:09.355845    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/447f2a14-e79a-4ffa-b257-0eb0d7cf82c7-metrics-certs\") pod \"maas-api-e2e-ipp-a-54370d-84b4f44586-66tqw\" (UID: \"447f2a14-e79a-4ffa-b257-0eb0d7cf82c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-54370d-84b4f44586-66tqw"
Sep 09 16:37:09.356011 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:09.355941    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/447f2a14-e79a-4ffa-b257-0eb0d7cf82c7-maas-api-tls\") pod \"maas-api-e2e-ipp-a-54370d-84b4f44586-66tqw\" (UID: \"447f2a14-e79a-4ffa-b257-0eb0d7cf82c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-54370d-84b4f44586-66tqw"
Sep 09 16:37:09.356103 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:09.356007    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/447f2a14-e79a-4ffa-b257-0eb0d7cf82c7-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-54370d-84b4f44586-66tqw\" (UID: \"447f2a14-e79a-4ffa-b257-0eb0d7cf82c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-54370d-84b4f44586-66tqw"
Sep 09 16:37:09.456702 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:09.456652    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djk2w\" (UniqueName: \"kubernetes.io/projected/447f2a14-e79a-4ffa-b257-0eb0d7cf82c7-kube-api-access-djk2w\") pod \"maas-api-e2e-ipp-a-54370d-84b4f44586-66tqw\" (UID: \"447f2a14-e79a-4ffa-b257-0eb0d7cf82c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-54370d-84b4f44586-66tqw"
Sep 09 16:37:09.456702 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:09.456714    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/447f2a14-e79a-4ffa-b257-0eb0d7cf82c7-metrics-certs\") pod \"maas-api-e2e-ipp-a-54370d-84b4f44586-66tqw\" (UID: \"447f2a14-e79a-4ffa-b257-0eb0d7cf82c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-54370d-84b4f44586-66tqw"
Sep 09 16:37:09.456927 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:09.456740    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/447f2a14-e79a-4ffa-b257-0eb0d7cf82c7-maas-api-tls\") pod \"maas-api-e2e-ipp-a-54370d-84b4f44586-66tqw\" (UID: \"447f2a14-e79a-4ffa-b257-0eb0d7cf82c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-54370d-84b4f44586-66tqw"
Sep 09 16:37:09.456927 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:09.456794    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/447f2a14-e79a-4ffa-b257-0eb0d7cf82c7-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-54370d-84b4f44586-66tqw\" (UID: \"447f2a14-e79a-4ffa-b257-0eb0d7cf82c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-54370d-84b4f44586-66tqw"
Sep 09 16:37:09.456927 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:37:09.456876    2567 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-54370d-serving-cert: secret "maas-api-e2e-ipp-a-54370d-serving-cert" not found
Sep 09 16:37:09.457028 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:37:09.456949    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/447f2a14-e79a-4ffa-b257-0eb0d7cf82c7-maas-api-tls podName:447f2a14-e79a-4ffa-b257-0eb0d7cf82c7 nodeName:}" failed. No retries permitted until 2026-09-09 16:37:09.956928132 +0000 UTC m=+1705.374778110 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/447f2a14-e79a-4ffa-b257-0eb0d7cf82c7-maas-api-tls") pod "maas-api-e2e-ipp-a-54370d-84b4f44586-66tqw" (UID: "447f2a14-e79a-4ffa-b257-0eb0d7cf82c7") : secret "maas-api-e2e-ipp-a-54370d-serving-cert" not found
Sep 09 16:37:09.457513 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:09.457411    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/447f2a14-e79a-4ffa-b257-0eb0d7cf82c7-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-54370d-84b4f44586-66tqw\" (UID: \"447f2a14-e79a-4ffa-b257-0eb0d7cf82c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-54370d-84b4f44586-66tqw"
Sep 09 16:37:09.459352 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:09.459329    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/447f2a14-e79a-4ffa-b257-0eb0d7cf82c7-metrics-certs\") pod \"maas-api-e2e-ipp-a-54370d-84b4f44586-66tqw\" (UID: \"447f2a14-e79a-4ffa-b257-0eb0d7cf82c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-54370d-84b4f44586-66tqw"
Sep 09 16:37:09.464787 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:09.464770    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djk2w\" (UniqueName: \"kubernetes.io/projected/447f2a14-e79a-4ffa-b257-0eb0d7cf82c7-kube-api-access-djk2w\") pod \"maas-api-e2e-ipp-a-54370d-84b4f44586-66tqw\" (UID: \"447f2a14-e79a-4ffa-b257-0eb0d7cf82c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-54370d-84b4f44586-66tqw"
Sep 09 16:37:09.963128 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:09.963086    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/447f2a14-e79a-4ffa-b257-0eb0d7cf82c7-maas-api-tls\") pod \"maas-api-e2e-ipp-a-54370d-84b4f44586-66tqw\" (UID: \"447f2a14-e79a-4ffa-b257-0eb0d7cf82c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-54370d-84b4f44586-66tqw"
Sep 09 16:37:09.966120 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:09.966081    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/447f2a14-e79a-4ffa-b257-0eb0d7cf82c7-maas-api-tls\") pod \"maas-api-e2e-ipp-a-54370d-84b4f44586-66tqw\" (UID: \"447f2a14-e79a-4ffa-b257-0eb0d7cf82c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-54370d-84b4f44586-66tqw"
Sep 09 16:37:10.147499 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:10.147442    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-54370d-84b4f44586-66tqw"
Sep 09 16:37:10.418929 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:10.418902    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-54370d-84b4f44586-66tqw"]
Sep 09 16:37:10.420919 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:37:10.420896    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod447f2a14_e79a_4ffa_b257_0eb0d7cf82c7.slice/crio-18a3223e6ba42f7784b83e20c1a41b5b7627566ae86386e49324d4d039722c3b WatchSource:0}: Error finding container 18a3223e6ba42f7784b83e20c1a41b5b7627566ae86386e49324d4d039722c3b: Status 404 returned error can't find the container with id 18a3223e6ba42f7784b83e20c1a41b5b7627566ae86386e49324d4d039722c3b
Sep 09 16:37:10.422267 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:10.422248    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 16:37:11.062658 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:11.062624    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-54370d-84b4f44586-66tqw" event={"ID":"447f2a14-e79a-4ffa-b257-0eb0d7cf82c7","Type":"ContainerStarted","Data":"18a3223e6ba42f7784b83e20c1a41b5b7627566ae86386e49324d4d039722c3b"}
Sep 09 16:37:12.069132 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:12.069090    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-54370d-84b4f44586-66tqw" event={"ID":"447f2a14-e79a-4ffa-b257-0eb0d7cf82c7","Type":"ContainerStarted","Data":"3cec36817d2da6826b67f4d6377114b24949c9a9d870fc203adf9bc9e6ad0833"}
Sep 09 16:37:12.069569 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:12.069215    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-54370d-84b4f44586-66tqw"
Sep 09 16:37:12.087297 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:12.087244    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-54370d-84b4f44586-66tqw" podStartSLOduration=2.095891366 podStartE2EDuration="3.087228187s" totalImagesPullingTime="991.336821ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 16:37:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 16:37:12.085029887 +0000 UTC m=+1707.502879886" watchObservedRunningTime="2026-09-09 16:37:12.087228187 +0000 UTC m=+1707.505078231"
Sep 09 16:37:18.078959 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:18.078928    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-54370d-84b4f44586-66tqw"
Sep 09 16:37:18.811498 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:18.811429    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:37:22.533862 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:22.533819    2567 patch_prober.go:28] interesting pod/e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": read tcp 10.134.0.2:52656->10.134.0.56:15021: read: connection reset by peer" start-of-body=
Sep 09 16:37:22.534240 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:22.533905    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp" podUID="4e5b5a13-128c-4713-bd42-fb9de49b5838" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": read tcp 10.134.0.2:52656->10.134.0.56:15021: read: connection reset by peer"
Sep 09 16:37:23.140385 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:23.140351    2567 generic.go:356] "Generic (PLEG): container finished" podID="4e5b5a13-128c-4713-bd42-fb9de49b5838" containerID="3ce26658914895da58fcd28521aad7b68fa1b079d73f7ecf995aced2d136a49f" exitCode=0
Sep 09 16:37:23.140590 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:23.140421    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp" event={"ID":"4e5b5a13-128c-4713-bd42-fb9de49b5838","Type":"ContainerDied","Data":"3ce26658914895da58fcd28521aad7b68fa1b079d73f7ecf995aced2d136a49f"}
Sep 09 16:37:23.140865 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:23.140851    2567 scope.go:117] "RemoveContainer" containerID="3ce26658914895da58fcd28521aad7b68fa1b079d73f7ecf995aced2d136a49f"
Sep 09 16:37:23.142043 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:23.141989    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:37:23.142154 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:23.142100    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:37:23.142214 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:23.142159    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:37:24.147975 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:24.147931    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp" event={"ID":"4e5b5a13-128c-4713-bd42-fb9de49b5838","Type":"ContainerStarted","Data":"b135fc9f941ce69aad9f0384bef61996f54a888c8144f6d17cab30b5c0bdd780"}
Sep 09 16:37:24.811415 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:24.811368    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp"
Sep 09 16:37:24.811641 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:24.811534    2567 patch_prober.go:28] interesting pod/e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Sep 09 16:37:24.811641 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:24.811597    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp" podUID="4e5b5a13-128c-4713-bd42-fb9de49b5838" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Sep 09 16:37:25.811563 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:25.811522    2567 patch_prober.go:28] interesting pod/e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Sep 09 16:37:25.812168 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:25.812138    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp" podUID="4e5b5a13-128c-4713-bd42-fb9de49b5838" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Sep 09 16:37:26.811522 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:26.811480    2567 patch_prober.go:28] interesting pod/e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Sep 09 16:37:26.811911 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:26.811868    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp" podUID="4e5b5a13-128c-4713-bd42-fb9de49b5838" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Sep 09 16:37:27.811846 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:27.811807    2567 patch_prober.go:28] interesting pod/e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Sep 09 16:37:27.812161 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:27.812132    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp" podUID="4e5b5a13-128c-4713-bd42-fb9de49b5838" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Sep 09 16:37:28.812192 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:28.812145    2567 patch_prober.go:28] interesting pod/e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Sep 09 16:37:28.812752 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:28.812223    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp" podUID="4e5b5a13-128c-4713-bd42-fb9de49b5838" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Sep 09 16:37:29.815594 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:29.815561    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp"
Sep 09 16:37:29.815980 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:29.815861    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp"
Sep 09 16:37:29.816660 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:29.816645    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp"
Sep 09 16:37:34.131343 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:34.131307    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr"]
Sep 09 16:37:34.134990 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:34.134966    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr"
Sep 09 16:37:34.137640 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:34.137617    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-3c6036-openshift-default-dockercfg-6nxzn\""
Sep 09 16:37:34.149378 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:34.149352    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr"]
Sep 09 16:37:34.304891 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:34.304853    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9759ccec-0bb9-473b-b461-f1e2fbdaa51a-istiod-ca-cert\") pod \"e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr\" (UID: \"9759ccec-0bb9-473b-b461-f1e2fbdaa51a\") " pod="openshift-ingress/e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr"
Sep 09 16:37:34.304891 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:34.304892    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stng9\" (UniqueName: \"kubernetes.io/projected/9759ccec-0bb9-473b-b461-f1e2fbdaa51a-kube-api-access-stng9\") pod \"e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr\" (UID: \"9759ccec-0bb9-473b-b461-f1e2fbdaa51a\") " pod="openshift-ingress/e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr"
Sep 09 16:37:34.305112 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:34.304955    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9759ccec-0bb9-473b-b461-f1e2fbdaa51a-istio-data\") pod \"e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr\" (UID: \"9759ccec-0bb9-473b-b461-f1e2fbdaa51a\") " pod="openshift-ingress/e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr"
Sep 09 16:37:34.305112 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:34.305013    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9759ccec-0bb9-473b-b461-f1e2fbdaa51a-istio-podinfo\") pod \"e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr\" (UID: \"9759ccec-0bb9-473b-b461-f1e2fbdaa51a\") " pod="openshift-ingress/e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr"
Sep 09 16:37:34.305112 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:34.305068    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9759ccec-0bb9-473b-b461-f1e2fbdaa51a-workload-socket\") pod \"e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr\" (UID: \"9759ccec-0bb9-473b-b461-f1e2fbdaa51a\") " pod="openshift-ingress/e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr"
Sep 09 16:37:34.305112 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:34.305085    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9759ccec-0bb9-473b-b461-f1e2fbdaa51a-istio-envoy\") pod \"e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr\" (UID: \"9759ccec-0bb9-473b-b461-f1e2fbdaa51a\") " pod="openshift-ingress/e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr"
Sep 09 16:37:34.305249 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:34.305136    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9759ccec-0bb9-473b-b461-f1e2fbdaa51a-istio-token\") pod \"e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr\" (UID: \"9759ccec-0bb9-473b-b461-f1e2fbdaa51a\") " pod="openshift-ingress/e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr"
Sep 09 16:37:34.305249 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:34.305159    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9759ccec-0bb9-473b-b461-f1e2fbdaa51a-workload-certs\") pod \"e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr\" (UID: \"9759ccec-0bb9-473b-b461-f1e2fbdaa51a\") " pod="openshift-ingress/e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr"
Sep 09 16:37:34.305249 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:34.305191    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9759ccec-0bb9-473b-b461-f1e2fbdaa51a-credential-socket\") pod \"e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr\" (UID: \"9759ccec-0bb9-473b-b461-f1e2fbdaa51a\") " pod="openshift-ingress/e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr"
Sep 09 16:37:34.405906 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:34.405818    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9759ccec-0bb9-473b-b461-f1e2fbdaa51a-workload-socket\") pod \"e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr\" (UID: \"9759ccec-0bb9-473b-b461-f1e2fbdaa51a\") " pod="openshift-ingress/e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr"
Sep 09 16:37:34.405906 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:34.405853    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9759ccec-0bb9-473b-b461-f1e2fbdaa51a-istio-envoy\") pod \"e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr\" (UID: \"9759ccec-0bb9-473b-b461-f1e2fbdaa51a\") " pod="openshift-ingress/e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr"
Sep 09 16:37:34.405906 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:34.405882    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9759ccec-0bb9-473b-b461-f1e2fbdaa51a-istio-token\") pod \"e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr\" (UID: \"9759ccec-0bb9-473b-b461-f1e2fbdaa51a\") " pod="openshift-ingress/e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr"
Sep 09 16:37:34.406252 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:34.405912    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9759ccec-0bb9-473b-b461-f1e2fbdaa51a-workload-certs\") pod \"e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr\" (UID: \"9759ccec-0bb9-473b-b461-f1e2fbdaa51a\") " pod="openshift-ingress/e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr"
Sep 09 16:37:34.406252 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:34.405949    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9759ccec-0bb9-473b-b461-f1e2fbdaa51a-credential-socket\") pod \"e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr\" (UID: \"9759ccec-0bb9-473b-b461-f1e2fbdaa51a\") " pod="openshift-ingress/e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr"
Sep 09 16:37:34.406252 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:34.405987    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9759ccec-0bb9-473b-b461-f1e2fbdaa51a-istiod-ca-cert\") pod \"e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr\" (UID: \"9759ccec-0bb9-473b-b461-f1e2fbdaa51a\") " pod="openshift-ingress/e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr"
Sep 09 16:37:34.406252 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:34.406019    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stng9\" (UniqueName: \"kubernetes.io/projected/9759ccec-0bb9-473b-b461-f1e2fbdaa51a-kube-api-access-stng9\") pod \"e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr\" (UID: \"9759ccec-0bb9-473b-b461-f1e2fbdaa51a\") " pod="openshift-ingress/e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr"
Sep 09 16:37:34.406252 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:34.406092    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9759ccec-0bb9-473b-b461-f1e2fbdaa51a-istio-data\") pod \"e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr\" (UID: \"9759ccec-0bb9-473b-b461-f1e2fbdaa51a\") " pod="openshift-ingress/e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr"
Sep 09 16:37:34.406252 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:34.406121    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9759ccec-0bb9-473b-b461-f1e2fbdaa51a-istio-podinfo\") pod \"e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr\" (UID: \"9759ccec-0bb9-473b-b461-f1e2fbdaa51a\") " pod="openshift-ingress/e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr"
Sep 09 16:37:34.406671 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:34.406269    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9759ccec-0bb9-473b-b461-f1e2fbdaa51a-workload-socket\") pod \"e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr\" (UID: \"9759ccec-0bb9-473b-b461-f1e2fbdaa51a\") " pod="openshift-ingress/e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr"
Sep 09 16:37:34.406671 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:34.406350    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9759ccec-0bb9-473b-b461-f1e2fbdaa51a-workload-certs\") pod \"e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr\" (UID: \"9759ccec-0bb9-473b-b461-f1e2fbdaa51a\") " pod="openshift-ingress/e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr"
Sep 09 16:37:34.406671 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:34.406644    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9759ccec-0bb9-473b-b461-f1e2fbdaa51a-istio-data\") pod \"e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr\" (UID: \"9759ccec-0bb9-473b-b461-f1e2fbdaa51a\") " pod="openshift-ingress/e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr"
Sep 09 16:37:34.406810 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:34.406729    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9759ccec-0bb9-473b-b461-f1e2fbdaa51a-istiod-ca-cert\") pod \"e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr\" (UID: \"9759ccec-0bb9-473b-b461-f1e2fbdaa51a\") " pod="openshift-ingress/e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr"
Sep 09 16:37:34.406810 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:34.406727    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9759ccec-0bb9-473b-b461-f1e2fbdaa51a-credential-socket\") pod \"e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr\" (UID: \"9759ccec-0bb9-473b-b461-f1e2fbdaa51a\") " pod="openshift-ingress/e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr"
Sep 09 16:37:34.408715 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:34.408690    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9759ccec-0bb9-473b-b461-f1e2fbdaa51a-istio-envoy\") pod \"e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr\" (UID: \"9759ccec-0bb9-473b-b461-f1e2fbdaa51a\") " pod="openshift-ingress/e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr"
Sep 09 16:37:34.408858 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:34.408832    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9759ccec-0bb9-473b-b461-f1e2fbdaa51a-istio-podinfo\") pod \"e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr\" (UID: \"9759ccec-0bb9-473b-b461-f1e2fbdaa51a\") " pod="openshift-ingress/e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr"
Sep 09 16:37:34.419690 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:34.419658    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9759ccec-0bb9-473b-b461-f1e2fbdaa51a-istio-token\") pod \"e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr\" (UID: \"9759ccec-0bb9-473b-b461-f1e2fbdaa51a\") " pod="openshift-ingress/e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr"
Sep 09 16:37:34.423877 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:34.423846    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-stng9\" (UniqueName: \"kubernetes.io/projected/9759ccec-0bb9-473b-b461-f1e2fbdaa51a-kube-api-access-stng9\") pod \"e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr\" (UID: \"9759ccec-0bb9-473b-b461-f1e2fbdaa51a\") " pod="openshift-ingress/e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr"
Sep 09 16:37:34.444290 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:34.444267    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr"
Sep 09 16:37:34.725577 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:34.725550    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr"]
Sep 09 16:37:34.727585 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:37:34.727562    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9759ccec_0bb9_473b_b461_f1e2fbdaa51a.slice/crio-7226d8fc7097d3a50b7d8d70e7bbadd6b8e62e4fffb22184e79cb5f6c37fb13d WatchSource:0}: Error finding container 7226d8fc7097d3a50b7d8d70e7bbadd6b8e62e4fffb22184e79cb5f6c37fb13d: Status 404 returned error can't find the container with id 7226d8fc7097d3a50b7d8d70e7bbadd6b8e62e4fffb22184e79cb5f6c37fb13d
Sep 09 16:37:34.729874 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:34.729835    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:37:34.729956 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:34.729905    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:37:34.729956 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:34.729943    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:37:35.201102 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:35.201066    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr" event={"ID":"9759ccec-0bb9-473b-b461-f1e2fbdaa51a","Type":"ContainerStarted","Data":"83768d4a8797d376deab3ca8464ea5dfa11e851656769cfde646c91322b2aefa"}
Sep 09 16:37:35.201532 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:35.201121    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr" event={"ID":"9759ccec-0bb9-473b-b461-f1e2fbdaa51a","Type":"ContainerStarted","Data":"7226d8fc7097d3a50b7d8d70e7bbadd6b8e62e4fffb22184e79cb5f6c37fb13d"}
Sep 09 16:37:35.221030 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:35.220946    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr" podStartSLOduration=1.220928985 podStartE2EDuration="1.220928985s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 16:37:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 16:37:35.219689838 +0000 UTC m=+1730.637539832" watchObservedRunningTime="2026-09-09 16:37:35.220928985 +0000 UTC m=+1730.638778985"
Sep 09 16:37:35.444540 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:35.444491    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr"
Sep 09 16:37:35.449906 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:35.449877    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr"
Sep 09 16:37:36.205640 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:36.205607    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr"
Sep 09 16:37:36.206534 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:36.206504    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr"
Sep 09 16:37:43.515186 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:43.515144    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:37:52.247648 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:52.247598    2567 patch_prober.go:28] interesting pod/e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 09 16:37:52.248044 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:52.247680    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr" podUID="9759ccec-0bb9-473b-b461-f1e2fbdaa51a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 09 16:37:53.295051 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:53.295016    2567 generic.go:356] "Generic (PLEG): container finished" podID="9759ccec-0bb9-473b-b461-f1e2fbdaa51a" containerID="83768d4a8797d376deab3ca8464ea5dfa11e851656769cfde646c91322b2aefa" exitCode=0
Sep 09 16:37:53.295437 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:53.295096    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr" event={"ID":"9759ccec-0bb9-473b-b461-f1e2fbdaa51a","Type":"ContainerDied","Data":"83768d4a8797d376deab3ca8464ea5dfa11e851656769cfde646c91322b2aefa"}
Sep 09 16:37:53.295571 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:53.295557    2567 scope.go:117] "RemoveContainer" containerID="83768d4a8797d376deab3ca8464ea5dfa11e851656769cfde646c91322b2aefa"
Sep 09 16:37:53.296853 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:53.296828    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:37:53.296925 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:53.296895    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:37:53.296962 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:53.296927    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 16:37:54.306473 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:54.306408    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr" event={"ID":"9759ccec-0bb9-473b-b461-f1e2fbdaa51a","Type":"ContainerStarted","Data":"b3e33a842f92ca955a7ee8f41e1d58855a99b08a57733fb2f6a376621c312c84"}
Sep 09 16:37:54.445108 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:54.445070    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr"
Sep 09 16:37:54.445293 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:54.445275    2567 patch_prober.go:28] interesting pod/e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Sep 09 16:37:54.445339 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:54.445317    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr" podUID="9759ccec-0bb9-473b-b461-f1e2fbdaa51a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Sep 09 16:37:55.445421 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:55.445379    2567 patch_prober.go:28] interesting pod/e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Sep 09 16:37:55.445944 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:55.445494    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr" podUID="9759ccec-0bb9-473b-b461-f1e2fbdaa51a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Sep 09 16:37:56.444880 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:56.444848    2567 patch_prober.go:28] interesting pod/e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Sep 09 16:37:56.445072 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:56.444904    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr" podUID="9759ccec-0bb9-473b-b461-f1e2fbdaa51a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Sep 09 16:37:57.445102 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:57.445057    2567 patch_prober.go:28] interesting pod/e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Sep 09 16:37:57.445526 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:57.445117    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr" podUID="9759ccec-0bb9-473b-b461-f1e2fbdaa51a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Sep 09 16:37:58.444816 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:58.444779    2567 patch_prober.go:28] interesting pod/e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Sep 09 16:37:58.445012 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:58.444837    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr" podUID="9759ccec-0bb9-473b-b461-f1e2fbdaa51a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Sep 09 16:37:59.445157 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:59.445117    2567 patch_prober.go:28] interesting pod/e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Sep 09 16:37:59.445567 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:37:59.445184    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr" podUID="9759ccec-0bb9-473b-b461-f1e2fbdaa51a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Sep 09 16:38:00.449008 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:38:00.448976    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr"
Sep 09 16:38:00.449411 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:38:00.449242    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr"
Sep 09 16:38:00.449889 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:38:00.449871    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr"
Sep 09 16:38:17.710780 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:38:17.710736    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:38:33.412704 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:38:33.412667    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:38:45.174236 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:38:45.174205    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-xwlzs_16ecfb4d-7944-465e-a38e-d8c5bda6b1ac/console-operator/2.log"
Sep 09 16:38:45.186732 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:38:45.186708    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-xwlzs_16ecfb4d-7944-465e-a38e-d8c5bda6b1ac/console-operator/2.log"
Sep 09 16:38:45.907394 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:38:45.907359    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-936acc-b94454c77-gbvns"]
Sep 09 16:38:45.910896 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:38:45.910868    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-936acc-b94454c77-gbvns"
Sep 09 16:38:45.913233 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:38:45.913209    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-936acc-serving-cert\""
Sep 09 16:38:45.920147 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:38:45.920122    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-936acc-b94454c77-gbvns"]
Sep 09 16:38:46.033601 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:38:46.033569    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99b1c6ad-1a67-40f5-a39d-3c23c6693e42-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-936acc-b94454c77-gbvns\" (UID: \"99b1c6ad-1a67-40f5-a39d-3c23c6693e42\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-936acc-b94454c77-gbvns"
Sep 09 16:38:46.033778 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:38:46.033609    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99b1c6ad-1a67-40f5-a39d-3c23c6693e42-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-936acc-b94454c77-gbvns\" (UID: \"99b1c6ad-1a67-40f5-a39d-3c23c6693e42\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-936acc-b94454c77-gbvns"
Sep 09 16:38:46.033778 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:38:46.033634    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/99b1c6ad-1a67-40f5-a39d-3c23c6693e42-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-936acc-b94454c77-gbvns\" (UID: \"99b1c6ad-1a67-40f5-a39d-3c23c6693e42\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-936acc-b94454c77-gbvns"
Sep 09 16:38:46.033778 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:38:46.033673    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64jtw\" (UniqueName: \"kubernetes.io/projected/99b1c6ad-1a67-40f5-a39d-3c23c6693e42-kube-api-access-64jtw\") pod \"maas-api-e2e-ipp-cleanup-936acc-b94454c77-gbvns\" (UID: \"99b1c6ad-1a67-40f5-a39d-3c23c6693e42\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-936acc-b94454c77-gbvns"
Sep 09 16:38:46.135241 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:38:46.135196    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99b1c6ad-1a67-40f5-a39d-3c23c6693e42-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-936acc-b94454c77-gbvns\" (UID: \"99b1c6ad-1a67-40f5-a39d-3c23c6693e42\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-936acc-b94454c77-gbvns"
Sep 09 16:38:46.135420 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:38:46.135286    2567 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-936acc-serving-cert: secret "maas-api-e2e-ipp-cleanup-936acc-serving-cert" not found
Sep 09 16:38:46.135420 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:38:46.135360    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99b1c6ad-1a67-40f5-a39d-3c23c6693e42-maas-api-tls podName:99b1c6ad-1a67-40f5-a39d-3c23c6693e42 nodeName:}" failed. No retries permitted until 2026-09-09 16:38:46.635338473 +0000 UTC m=+1802.053188449 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/99b1c6ad-1a67-40f5-a39d-3c23c6693e42-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-936acc-b94454c77-gbvns" (UID: "99b1c6ad-1a67-40f5-a39d-3c23c6693e42") : secret "maas-api-e2e-ipp-cleanup-936acc-serving-cert" not found
Sep 09 16:38:46.135562 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:38:46.135414    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99b1c6ad-1a67-40f5-a39d-3c23c6693e42-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-936acc-b94454c77-gbvns\" (UID: \"99b1c6ad-1a67-40f5-a39d-3c23c6693e42\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-936acc-b94454c77-gbvns"
Sep 09 16:38:46.135562 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:38:46.135450    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/99b1c6ad-1a67-40f5-a39d-3c23c6693e42-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-936acc-b94454c77-gbvns\" (UID: \"99b1c6ad-1a67-40f5-a39d-3c23c6693e42\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-936acc-b94454c77-gbvns"
Sep 09 16:38:46.135562 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:38:46.135510    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64jtw\" (UniqueName: \"kubernetes.io/projected/99b1c6ad-1a67-40f5-a39d-3c23c6693e42-kube-api-access-64jtw\") pod \"maas-api-e2e-ipp-cleanup-936acc-b94454c77-gbvns\" (UID: \"99b1c6ad-1a67-40f5-a39d-3c23c6693e42\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-936acc-b94454c77-gbvns"
Sep 09 16:38:46.136028 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:38:46.136009    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/99b1c6ad-1a67-40f5-a39d-3c23c6693e42-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-936acc-b94454c77-gbvns\" (UID: \"99b1c6ad-1a67-40f5-a39d-3c23c6693e42\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-936acc-b94454c77-gbvns"
Sep 09 16:38:46.138169 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:38:46.138149    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99b1c6ad-1a67-40f5-a39d-3c23c6693e42-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-936acc-b94454c77-gbvns\" (UID: \"99b1c6ad-1a67-40f5-a39d-3c23c6693e42\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-936acc-b94454c77-gbvns"
Sep 09 16:38:46.144063 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:38:46.144036    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-64jtw\" (UniqueName: \"kubernetes.io/projected/99b1c6ad-1a67-40f5-a39d-3c23c6693e42-kube-api-access-64jtw\") pod \"maas-api-e2e-ipp-cleanup-936acc-b94454c77-gbvns\" (UID: \"99b1c6ad-1a67-40f5-a39d-3c23c6693e42\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-936acc-b94454c77-gbvns"
Sep 09 16:38:46.640832 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:38:46.640784    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99b1c6ad-1a67-40f5-a39d-3c23c6693e42-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-936acc-b94454c77-gbvns\" (UID: \"99b1c6ad-1a67-40f5-a39d-3c23c6693e42\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-936acc-b94454c77-gbvns"
Sep 09 16:38:46.643878 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:38:46.643849    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99b1c6ad-1a67-40f5-a39d-3c23c6693e42-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-936acc-b94454c77-gbvns\" (UID: \"99b1c6ad-1a67-40f5-a39d-3c23c6693e42\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-936acc-b94454c77-gbvns"
Sep 09 16:38:46.826505 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:38:46.826439    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-936acc-b94454c77-gbvns"
Sep 09 16:38:47.101900 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:38:47.101872    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-936acc-b94454c77-gbvns"]
Sep 09 16:38:47.104175 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:38:47.104141    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99b1c6ad_1a67_40f5_a39d_3c23c6693e42.slice/crio-dd5d94478381c59bf27001ecbff3d6dacd13a798a6dd5c2b207ab5f4b1acb937 WatchSource:0}: Error finding container dd5d94478381c59bf27001ecbff3d6dacd13a798a6dd5c2b207ab5f4b1acb937: Status 404 returned error can't find the container with id dd5d94478381c59bf27001ecbff3d6dacd13a798a6dd5c2b207ab5f4b1acb937
Sep 09 16:38:47.579824 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:38:47.579784    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-936acc-b94454c77-gbvns" event={"ID":"99b1c6ad-1a67-40f5-a39d-3c23c6693e42","Type":"ContainerStarted","Data":"dd5d94478381c59bf27001ecbff3d6dacd13a798a6dd5c2b207ab5f4b1acb937"}
Sep 09 16:38:48.586092 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:38:48.586059    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-936acc-b94454c77-gbvns" event={"ID":"99b1c6ad-1a67-40f5-a39d-3c23c6693e42","Type":"ContainerStarted","Data":"cb6938a4db612ca146d19fd1a8e8a8f6f4a3732b403c8afc4d95adf8ae8e8dd3"}
Sep 09 16:38:48.586512 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:38:48.586146    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-936acc-b94454c77-gbvns"
Sep 09 16:38:48.604182 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:38:48.604137    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-936acc-b94454c77-gbvns" podStartSLOduration=2.513836787 podStartE2EDuration="3.604120822s" totalImagesPullingTime="1.090284035s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 16:38:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 16:38:48.601363513 +0000 UTC m=+1804.019213504" watchObservedRunningTime="2026-09-09 16:38:48.604120822 +0000 UTC m=+1804.021970821"
Sep 09 16:38:53.411982 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:38:53.411938    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:38:54.596770 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:38:54.596740    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-936acc-b94454c77-gbvns"
Sep 09 16:39:16.542523 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:39:16.542489    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-936acc-b94454c77-gbvns"]
Sep 09 16:39:16.543092 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:39:16.542750    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-936acc-b94454c77-gbvns" podUID="99b1c6ad-1a67-40f5-a39d-3c23c6693e42" containerName="maas-api" containerID="cri-o://cb6938a4db612ca146d19fd1a8e8a8f6f4a3732b403c8afc4d95adf8ae8e8dd3" gracePeriod=30
Sep 09 16:39:16.733584 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:39:16.733553    2567 generic.go:356] "Generic (PLEG): container finished" podID="99b1c6ad-1a67-40f5-a39d-3c23c6693e42" containerID="cb6938a4db612ca146d19fd1a8e8a8f6f4a3732b403c8afc4d95adf8ae8e8dd3" exitCode=0
Sep 09 16:39:16.733747 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:39:16.733680    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-936acc-b94454c77-gbvns" event={"ID":"99b1c6ad-1a67-40f5-a39d-3c23c6693e42","Type":"ContainerDied","Data":"cb6938a4db612ca146d19fd1a8e8a8f6f4a3732b403c8afc4d95adf8ae8e8dd3"}
Sep 09 16:39:16.801834 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:39:16.801766    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-936acc-b94454c77-gbvns"
Sep 09 16:39:16.945797 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:39:16.945757    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/99b1c6ad-1a67-40f5-a39d-3c23c6693e42-openshift-service-ca\") pod \"99b1c6ad-1a67-40f5-a39d-3c23c6693e42\" (UID: \"99b1c6ad-1a67-40f5-a39d-3c23c6693e42\") "
Sep 09 16:39:16.946003 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:39:16.945865    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-64jtw\" (UniqueName: \"kubernetes.io/projected/99b1c6ad-1a67-40f5-a39d-3c23c6693e42-kube-api-access-64jtw\") pod \"99b1c6ad-1a67-40f5-a39d-3c23c6693e42\" (UID: \"99b1c6ad-1a67-40f5-a39d-3c23c6693e42\") "
Sep 09 16:39:16.946003 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:39:16.945905    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99b1c6ad-1a67-40f5-a39d-3c23c6693e42-maas-api-tls\") pod \"99b1c6ad-1a67-40f5-a39d-3c23c6693e42\" (UID: \"99b1c6ad-1a67-40f5-a39d-3c23c6693e42\") "
Sep 09 16:39:16.946003 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:39:16.945998    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99b1c6ad-1a67-40f5-a39d-3c23c6693e42-metrics-certs\") pod \"99b1c6ad-1a67-40f5-a39d-3c23c6693e42\" (UID: \"99b1c6ad-1a67-40f5-a39d-3c23c6693e42\") "
Sep 09 16:39:16.946245 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:39:16.946205    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99b1c6ad-1a67-40f5-a39d-3c23c6693e42-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "99b1c6ad-1a67-40f5-a39d-3c23c6693e42" (UID: "99b1c6ad-1a67-40f5-a39d-3c23c6693e42"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 16:39:16.946667 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:39:16.946369    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/99b1c6ad-1a67-40f5-a39d-3c23c6693e42-openshift-service-ca\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:39:16.951554 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:39:16.951499    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99b1c6ad-1a67-40f5-a39d-3c23c6693e42-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "99b1c6ad-1a67-40f5-a39d-3c23c6693e42" (UID: "99b1c6ad-1a67-40f5-a39d-3c23c6693e42"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 16:39:16.951702 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:39:16.951551    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99b1c6ad-1a67-40f5-a39d-3c23c6693e42-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "99b1c6ad-1a67-40f5-a39d-3c23c6693e42" (UID: "99b1c6ad-1a67-40f5-a39d-3c23c6693e42"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 16:39:16.954193 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:39:16.954147    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99b1c6ad-1a67-40f5-a39d-3c23c6693e42-kube-api-access-64jtw" (OuterVolumeSpecName: "kube-api-access-64jtw") pod "99b1c6ad-1a67-40f5-a39d-3c23c6693e42" (UID: "99b1c6ad-1a67-40f5-a39d-3c23c6693e42"). InnerVolumeSpecName "kube-api-access-64jtw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 16:39:17.047808 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:39:17.047782    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99b1c6ad-1a67-40f5-a39d-3c23c6693e42-metrics-certs\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:39:17.047808 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:39:17.047808    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-64jtw\" (UniqueName: \"kubernetes.io/projected/99b1c6ad-1a67-40f5-a39d-3c23c6693e42-kube-api-access-64jtw\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:39:17.047963 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:39:17.047817    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99b1c6ad-1a67-40f5-a39d-3c23c6693e42-maas-api-tls\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:39:17.739634 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:39:17.739602    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-936acc-b94454c77-gbvns" event={"ID":"99b1c6ad-1a67-40f5-a39d-3c23c6693e42","Type":"ContainerDied","Data":"dd5d94478381c59bf27001ecbff3d6dacd13a798a6dd5c2b207ab5f4b1acb937"}
Sep 09 16:39:17.740061 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:39:17.739661    2567 scope.go:117] "RemoveContainer" containerID="cb6938a4db612ca146d19fd1a8e8a8f6f4a3732b403c8afc4d95adf8ae8e8dd3"
Sep 09 16:39:17.740061 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:39:17.739608    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-936acc-b94454c77-gbvns"
Sep 09 16:39:17.756581 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:39:17.756559    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-936acc-b94454c77-gbvns"]
Sep 09 16:39:17.762520 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:39:17.762498    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-936acc-b94454c77-gbvns"]
Sep 09 16:39:19.036742 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:39:19.036707    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99b1c6ad-1a67-40f5-a39d-3c23c6693e42" path="/var/lib/kubelet/pods/99b1c6ad-1a67-40f5-a39d-3c23c6693e42/volumes"
Sep 09 16:39:21.309844 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:39:21.309808    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:39:40.203912 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:39:40.203873    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:39:42.577919 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:39:42.577885    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-54370d-84b4f44586-66tqw"]
Sep 09 16:39:42.578344 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:39:42.578147    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-54370d-84b4f44586-66tqw" podUID="447f2a14-e79a-4ffa-b257-0eb0d7cf82c7" containerName="maas-api" containerID="cri-o://3cec36817d2da6826b67f4d6377114b24949c9a9d870fc203adf9bc9e6ad0833" gracePeriod=30
Sep 09 16:39:42.844222 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:39:42.844191    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-54370d-84b4f44586-66tqw"
Sep 09 16:39:42.868506 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:39:42.868473    2567 generic.go:356] "Generic (PLEG): container finished" podID="447f2a14-e79a-4ffa-b257-0eb0d7cf82c7" containerID="3cec36817d2da6826b67f4d6377114b24949c9a9d870fc203adf9bc9e6ad0833" exitCode=0
Sep 09 16:39:42.868672 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:39:42.868521    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-54370d-84b4f44586-66tqw"
Sep 09 16:39:42.868672 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:39:42.868545    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-54370d-84b4f44586-66tqw" event={"ID":"447f2a14-e79a-4ffa-b257-0eb0d7cf82c7","Type":"ContainerDied","Data":"3cec36817d2da6826b67f4d6377114b24949c9a9d870fc203adf9bc9e6ad0833"}
Sep 09 16:39:42.868672 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:39:42.868599    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-54370d-84b4f44586-66tqw" event={"ID":"447f2a14-e79a-4ffa-b257-0eb0d7cf82c7","Type":"ContainerDied","Data":"18a3223e6ba42f7784b83e20c1a41b5b7627566ae86386e49324d4d039722c3b"}
Sep 09 16:39:42.868672 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:39:42.868621    2567 scope.go:117] "RemoveContainer" containerID="3cec36817d2da6826b67f4d6377114b24949c9a9d870fc203adf9bc9e6ad0833"
Sep 09 16:39:42.877865 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:39:42.877841    2567 scope.go:117] "RemoveContainer" containerID="3cec36817d2da6826b67f4d6377114b24949c9a9d870fc203adf9bc9e6ad0833"
Sep 09 16:39:42.878152 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:39:42.878133    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3cec36817d2da6826b67f4d6377114b24949c9a9d870fc203adf9bc9e6ad0833\": container with ID starting with 3cec36817d2da6826b67f4d6377114b24949c9a9d870fc203adf9bc9e6ad0833 not found: ID does not exist" containerID="3cec36817d2da6826b67f4d6377114b24949c9a9d870fc203adf9bc9e6ad0833"
Sep 09 16:39:42.878207 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:39:42.878163    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3cec36817d2da6826b67f4d6377114b24949c9a9d870fc203adf9bc9e6ad0833"} err="failed to get container status \"3cec36817d2da6826b67f4d6377114b24949c9a9d870fc203adf9bc9e6ad0833\": rpc error: code = NotFound desc = could not find container \"3cec36817d2da6826b67f4d6377114b24949c9a9d870fc203adf9bc9e6ad0833\": container with ID starting with 3cec36817d2da6826b67f4d6377114b24949c9a9d870fc203adf9bc9e6ad0833 not found: ID does not exist"
Sep 09 16:39:43.006638 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:39:43.006607    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/447f2a14-e79a-4ffa-b257-0eb0d7cf82c7-maas-api-tls\") pod \"447f2a14-e79a-4ffa-b257-0eb0d7cf82c7\" (UID: \"447f2a14-e79a-4ffa-b257-0eb0d7cf82c7\") "
Sep 09 16:39:43.006807 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:39:43.006741    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-djk2w\" (UniqueName: \"kubernetes.io/projected/447f2a14-e79a-4ffa-b257-0eb0d7cf82c7-kube-api-access-djk2w\") pod \"447f2a14-e79a-4ffa-b257-0eb0d7cf82c7\" (UID: \"447f2a14-e79a-4ffa-b257-0eb0d7cf82c7\") "
Sep 09 16:39:43.006807 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:39:43.006788    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/447f2a14-e79a-4ffa-b257-0eb0d7cf82c7-metrics-certs\") pod \"447f2a14-e79a-4ffa-b257-0eb0d7cf82c7\" (UID: \"447f2a14-e79a-4ffa-b257-0eb0d7cf82c7\") "
Sep 09 16:39:43.006903 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:39:43.006812    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/447f2a14-e79a-4ffa-b257-0eb0d7cf82c7-openshift-service-ca\") pod \"447f2a14-e79a-4ffa-b257-0eb0d7cf82c7\" (UID: \"447f2a14-e79a-4ffa-b257-0eb0d7cf82c7\") "
Sep 09 16:39:43.007302 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:39:43.007269    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/447f2a14-e79a-4ffa-b257-0eb0d7cf82c7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "447f2a14-e79a-4ffa-b257-0eb0d7cf82c7" (UID: "447f2a14-e79a-4ffa-b257-0eb0d7cf82c7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 16:39:43.008960 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:39:43.008932    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/447f2a14-e79a-4ffa-b257-0eb0d7cf82c7-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "447f2a14-e79a-4ffa-b257-0eb0d7cf82c7" (UID: "447f2a14-e79a-4ffa-b257-0eb0d7cf82c7"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 16:39:43.009147 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:39:43.009120    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/447f2a14-e79a-4ffa-b257-0eb0d7cf82c7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "447f2a14-e79a-4ffa-b257-0eb0d7cf82c7" (UID: "447f2a14-e79a-4ffa-b257-0eb0d7cf82c7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 16:39:43.009484 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:39:43.009443    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/447f2a14-e79a-4ffa-b257-0eb0d7cf82c7-kube-api-access-djk2w" (OuterVolumeSpecName: "kube-api-access-djk2w") pod "447f2a14-e79a-4ffa-b257-0eb0d7cf82c7" (UID: "447f2a14-e79a-4ffa-b257-0eb0d7cf82c7"). InnerVolumeSpecName "kube-api-access-djk2w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 16:39:43.108534 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:39:43.108426    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-djk2w\" (UniqueName: \"kubernetes.io/projected/447f2a14-e79a-4ffa-b257-0eb0d7cf82c7-kube-api-access-djk2w\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:39:43.108534 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:39:43.108482    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/447f2a14-e79a-4ffa-b257-0eb0d7cf82c7-metrics-certs\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:39:43.108534 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:39:43.108497    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/447f2a14-e79a-4ffa-b257-0eb0d7cf82c7-openshift-service-ca\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:39:43.108534 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:39:43.108508    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/447f2a14-e79a-4ffa-b257-0eb0d7cf82c7-maas-api-tls\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:39:43.188631 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:39:43.188596    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-54370d-84b4f44586-66tqw"]
Sep 09 16:39:43.192058 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:39:43.192030    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-54370d-84b4f44586-66tqw"]
Sep 09 16:39:45.038018 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:39:45.037985    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="447f2a14-e79a-4ffa-b257-0eb0d7cf82c7" path="/var/lib/kubelet/pods/447f2a14-e79a-4ffa-b257-0eb0d7cf82c7/volumes"
Sep 09 16:39:53.305170 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:39:53.305136    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:40:30.914174 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:30.914140    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp"]
Sep 09 16:40:30.914790 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:30.914374    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp" podUID="4e5b5a13-128c-4713-bd42-fb9de49b5838" containerName="istio-proxy" containerID="cri-o://b135fc9f941ce69aad9f0384bef61996f54a888c8144f6d17cab30b5c0bdd780" gracePeriod=30
Sep 09 16:40:36.135389 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:36.135353    2567 generic.go:356] "Generic (PLEG): container finished" podID="4e5b5a13-128c-4713-bd42-fb9de49b5838" containerID="b135fc9f941ce69aad9f0384bef61996f54a888c8144f6d17cab30b5c0bdd780" exitCode=0
Sep 09 16:40:36.135809 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:36.135392    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp" event={"ID":"4e5b5a13-128c-4713-bd42-fb9de49b5838","Type":"ContainerDied","Data":"b135fc9f941ce69aad9f0384bef61996f54a888c8144f6d17cab30b5c0bdd780"}
Sep 09 16:40:36.135809 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:36.135490    2567 scope.go:117] "RemoveContainer" containerID="3ce26658914895da58fcd28521aad7b68fa1b079d73f7ecf995aced2d136a49f"
Sep 09 16:40:36.169973 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:36.169949    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp"
Sep 09 16:40:36.229841 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:36.229764    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e5b5a13-128c-4713-bd42-fb9de49b5838-workload-certs\") pod \"4e5b5a13-128c-4713-bd42-fb9de49b5838\" (UID: \"4e5b5a13-128c-4713-bd42-fb9de49b5838\") "
Sep 09 16:40:36.229841 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:36.229817    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e5b5a13-128c-4713-bd42-fb9de49b5838-istio-envoy\") pod \"4e5b5a13-128c-4713-bd42-fb9de49b5838\" (UID: \"4e5b5a13-128c-4713-bd42-fb9de49b5838\") "
Sep 09 16:40:36.230047 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:36.229852    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e5b5a13-128c-4713-bd42-fb9de49b5838-istio-data\") pod \"4e5b5a13-128c-4713-bd42-fb9de49b5838\" (UID: \"4e5b5a13-128c-4713-bd42-fb9de49b5838\") "
Sep 09 16:40:36.230047 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:36.229882    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e5b5a13-128c-4713-bd42-fb9de49b5838-workload-socket\") pod \"4e5b5a13-128c-4713-bd42-fb9de49b5838\" (UID: \"4e5b5a13-128c-4713-bd42-fb9de49b5838\") "
Sep 09 16:40:36.230047 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:36.229931    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e5b5a13-128c-4713-bd42-fb9de49b5838-istio-token\") pod \"4e5b5a13-128c-4713-bd42-fb9de49b5838\" (UID: \"4e5b5a13-128c-4713-bd42-fb9de49b5838\") "
Sep 09 16:40:36.230047 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:36.229964    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e5b5a13-128c-4713-bd42-fb9de49b5838-credential-socket\") pod \"4e5b5a13-128c-4713-bd42-fb9de49b5838\" (UID: \"4e5b5a13-128c-4713-bd42-fb9de49b5838\") "
Sep 09 16:40:36.230047 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:36.230015    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e5b5a13-128c-4713-bd42-fb9de49b5838-istiod-ca-cert\") pod \"4e5b5a13-128c-4713-bd42-fb9de49b5838\" (UID: \"4e5b5a13-128c-4713-bd42-fb9de49b5838\") "
Sep 09 16:40:36.230292 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:36.230079    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e5b5a13-128c-4713-bd42-fb9de49b5838-istio-podinfo\") pod \"4e5b5a13-128c-4713-bd42-fb9de49b5838\" (UID: \"4e5b5a13-128c-4713-bd42-fb9de49b5838\") "
Sep 09 16:40:36.230292 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:36.230086    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e5b5a13-128c-4713-bd42-fb9de49b5838-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4e5b5a13-128c-4713-bd42-fb9de49b5838" (UID: "4e5b5a13-128c-4713-bd42-fb9de49b5838"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:40:36.230292 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:36.230172    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9khtq\" (UniqueName: \"kubernetes.io/projected/4e5b5a13-128c-4713-bd42-fb9de49b5838-kube-api-access-9khtq\") pod \"4e5b5a13-128c-4713-bd42-fb9de49b5838\" (UID: \"4e5b5a13-128c-4713-bd42-fb9de49b5838\") "
Sep 09 16:40:36.230292 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:36.230202    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e5b5a13-128c-4713-bd42-fb9de49b5838-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4e5b5a13-128c-4713-bd42-fb9de49b5838" (UID: "4e5b5a13-128c-4713-bd42-fb9de49b5838"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:40:36.230292 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:36.230259    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e5b5a13-128c-4713-bd42-fb9de49b5838-istio-data" (OuterVolumeSpecName: "istio-data") pod "4e5b5a13-128c-4713-bd42-fb9de49b5838" (UID: "4e5b5a13-128c-4713-bd42-fb9de49b5838"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:40:36.230616 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:36.230313    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e5b5a13-128c-4713-bd42-fb9de49b5838-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4e5b5a13-128c-4713-bd42-fb9de49b5838" (UID: "4e5b5a13-128c-4713-bd42-fb9de49b5838"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:40:36.230616 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:36.230563    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e5b5a13-128c-4713-bd42-fb9de49b5838-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4e5b5a13-128c-4713-bd42-fb9de49b5838" (UID: "4e5b5a13-128c-4713-bd42-fb9de49b5838"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 16:40:36.230616 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:36.230581    2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e5b5a13-128c-4713-bd42-fb9de49b5838-workload-certs\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:40:36.230616 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:36.230602    2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e5b5a13-128c-4713-bd42-fb9de49b5838-istio-data\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:40:36.230616 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:36.230616    2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e5b5a13-128c-4713-bd42-fb9de49b5838-workload-socket\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:40:36.231057 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:36.230629    2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e5b5a13-128c-4713-bd42-fb9de49b5838-credential-socket\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:40:36.232301 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:36.232277    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4e5b5a13-128c-4713-bd42-fb9de49b5838-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4e5b5a13-128c-4713-bd42-fb9de49b5838" (UID: "4e5b5a13-128c-4713-bd42-fb9de49b5838"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 16:40:36.232399 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:36.232300    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e5b5a13-128c-4713-bd42-fb9de49b5838-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4e5b5a13-128c-4713-bd42-fb9de49b5838" (UID: "4e5b5a13-128c-4713-bd42-fb9de49b5838"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:40:36.232399 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:36.232346    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e5b5a13-128c-4713-bd42-fb9de49b5838-istio-token" (OuterVolumeSpecName: "istio-token") pod "4e5b5a13-128c-4713-bd42-fb9de49b5838" (UID: "4e5b5a13-128c-4713-bd42-fb9de49b5838"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 16:40:36.232565 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:36.232548    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e5b5a13-128c-4713-bd42-fb9de49b5838-kube-api-access-9khtq" (OuterVolumeSpecName: "kube-api-access-9khtq") pod "4e5b5a13-128c-4713-bd42-fb9de49b5838" (UID: "4e5b5a13-128c-4713-bd42-fb9de49b5838"). InnerVolumeSpecName "kube-api-access-9khtq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 16:40:36.331230 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:36.331193    2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e5b5a13-128c-4713-bd42-fb9de49b5838-istiod-ca-cert\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:40:36.331230 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:36.331222    2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e5b5a13-128c-4713-bd42-fb9de49b5838-istio-podinfo\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:40:36.331230 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:36.331231    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9khtq\" (UniqueName: \"kubernetes.io/projected/4e5b5a13-128c-4713-bd42-fb9de49b5838-kube-api-access-9khtq\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:40:36.331230 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:36.331239    2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e5b5a13-128c-4713-bd42-fb9de49b5838-istio-envoy\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:40:36.331531 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:36.331249    2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e5b5a13-128c-4713-bd42-fb9de49b5838-istio-token\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:40:37.141167 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:37.141135    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp"
Sep 09 16:40:37.141167 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:37.141146    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp" event={"ID":"4e5b5a13-128c-4713-bd42-fb9de49b5838","Type":"ContainerDied","Data":"f4567842d6b8d636cc89df002961ef50cc1635d77b93a00ae41e7ee3d62e99aa"}
Sep 09 16:40:37.141723 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:37.141198    2567 scope.go:117] "RemoveContainer" containerID="b135fc9f941ce69aad9f0384bef61996f54a888c8144f6d17cab30b5c0bdd780"
Sep 09 16:40:37.157872 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:37.157846    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp"]
Sep 09 16:40:37.163937 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:37.163915    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-54370d-openshift-default-696487bf76-j8vlp"]
Sep 09 16:40:39.036091 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:39.036052    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e5b5a13-128c-4713-bd42-fb9de49b5838" path="/var/lib/kubelet/pods/4e5b5a13-128c-4713-bd42-fb9de49b5838/volumes"
Sep 09 16:40:44.907162 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:44.907123    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:40:48.334176 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:48.334141    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr"]
Sep 09 16:40:48.334694 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:48.334471    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr" podUID="9759ccec-0bb9-473b-b461-f1e2fbdaa51a" containerName="istio-proxy" containerID="cri-o://b3e33a842f92ca955a7ee8f41e1d58855a99b08a57733fb2f6a376621c312c84" gracePeriod=30
Sep 09 16:40:53.587695 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:53.587670    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr"
Sep 09 16:40:53.690932 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:53.690836    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9759ccec-0bb9-473b-b461-f1e2fbdaa51a-istio-podinfo\") pod \"9759ccec-0bb9-473b-b461-f1e2fbdaa51a\" (UID: \"9759ccec-0bb9-473b-b461-f1e2fbdaa51a\") "
Sep 09 16:40:53.690932 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:53.690880    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9759ccec-0bb9-473b-b461-f1e2fbdaa51a-credential-socket\") pod \"9759ccec-0bb9-473b-b461-f1e2fbdaa51a\" (UID: \"9759ccec-0bb9-473b-b461-f1e2fbdaa51a\") "
Sep 09 16:40:53.690932 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:53.690922    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9759ccec-0bb9-473b-b461-f1e2fbdaa51a-istio-token\") pod \"9759ccec-0bb9-473b-b461-f1e2fbdaa51a\" (UID: \"9759ccec-0bb9-473b-b461-f1e2fbdaa51a\") "
Sep 09 16:40:53.691225 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:53.690945    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9759ccec-0bb9-473b-b461-f1e2fbdaa51a-workload-certs\") pod \"9759ccec-0bb9-473b-b461-f1e2fbdaa51a\" (UID: \"9759ccec-0bb9-473b-b461-f1e2fbdaa51a\") "
Sep 09 16:40:53.691225 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:53.690982    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9759ccec-0bb9-473b-b461-f1e2fbdaa51a-workload-socket\") pod \"9759ccec-0bb9-473b-b461-f1e2fbdaa51a\" (UID: \"9759ccec-0bb9-473b-b461-f1e2fbdaa51a\") "
Sep 09 16:40:53.691225 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:53.691021    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9759ccec-0bb9-473b-b461-f1e2fbdaa51a-istiod-ca-cert\") pod \"9759ccec-0bb9-473b-b461-f1e2fbdaa51a\" (UID: \"9759ccec-0bb9-473b-b461-f1e2fbdaa51a\") "
Sep 09 16:40:53.691225 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:53.691044    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-stng9\" (UniqueName: \"kubernetes.io/projected/9759ccec-0bb9-473b-b461-f1e2fbdaa51a-kube-api-access-stng9\") pod \"9759ccec-0bb9-473b-b461-f1e2fbdaa51a\" (UID: \"9759ccec-0bb9-473b-b461-f1e2fbdaa51a\") "
Sep 09 16:40:53.691225 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:53.691066    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9759ccec-0bb9-473b-b461-f1e2fbdaa51a-istio-data\") pod \"9759ccec-0bb9-473b-b461-f1e2fbdaa51a\" (UID: \"9759ccec-0bb9-473b-b461-f1e2fbdaa51a\") "
Sep 09 16:40:53.691225 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:53.691108    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9759ccec-0bb9-473b-b461-f1e2fbdaa51a-istio-envoy\") pod \"9759ccec-0bb9-473b-b461-f1e2fbdaa51a\" (UID: \"9759ccec-0bb9-473b-b461-f1e2fbdaa51a\") "
Sep 09 16:40:53.691638 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:53.691252    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9759ccec-0bb9-473b-b461-f1e2fbdaa51a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9759ccec-0bb9-473b-b461-f1e2fbdaa51a" (UID: "9759ccec-0bb9-473b-b461-f1e2fbdaa51a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:40:53.691638 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:53.691320    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9759ccec-0bb9-473b-b461-f1e2fbdaa51a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9759ccec-0bb9-473b-b461-f1e2fbdaa51a" (UID: "9759ccec-0bb9-473b-b461-f1e2fbdaa51a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:40:53.691638 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:53.691502    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9759ccec-0bb9-473b-b461-f1e2fbdaa51a-istio-data" (OuterVolumeSpecName: "istio-data") pod "9759ccec-0bb9-473b-b461-f1e2fbdaa51a" (UID: "9759ccec-0bb9-473b-b461-f1e2fbdaa51a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:40:53.691638 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:53.691514    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9759ccec-0bb9-473b-b461-f1e2fbdaa51a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9759ccec-0bb9-473b-b461-f1e2fbdaa51a" (UID: "9759ccec-0bb9-473b-b461-f1e2fbdaa51a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:40:53.691638 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:53.691443    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9759ccec-0bb9-473b-b461-f1e2fbdaa51a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9759ccec-0bb9-473b-b461-f1e2fbdaa51a" (UID: "9759ccec-0bb9-473b-b461-f1e2fbdaa51a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 16:40:53.691638 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:53.691607    2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9759ccec-0bb9-473b-b461-f1e2fbdaa51a-workload-certs\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:40:53.691900 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:53.691644    2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9759ccec-0bb9-473b-b461-f1e2fbdaa51a-credential-socket\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:40:53.693529 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:53.693495    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9759ccec-0bb9-473b-b461-f1e2fbdaa51a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9759ccec-0bb9-473b-b461-f1e2fbdaa51a" (UID: "9759ccec-0bb9-473b-b461-f1e2fbdaa51a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:40:53.693655 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:53.693612    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9759ccec-0bb9-473b-b461-f1e2fbdaa51a-kube-api-access-stng9" (OuterVolumeSpecName: "kube-api-access-stng9") pod "9759ccec-0bb9-473b-b461-f1e2fbdaa51a" (UID: "9759ccec-0bb9-473b-b461-f1e2fbdaa51a"). InnerVolumeSpecName "kube-api-access-stng9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 16:40:53.693807 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:53.693782    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9759ccec-0bb9-473b-b461-f1e2fbdaa51a-istio-token" (OuterVolumeSpecName: "istio-token") pod "9759ccec-0bb9-473b-b461-f1e2fbdaa51a" (UID: "9759ccec-0bb9-473b-b461-f1e2fbdaa51a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 16:40:53.693807 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:53.693787    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9759ccec-0bb9-473b-b461-f1e2fbdaa51a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9759ccec-0bb9-473b-b461-f1e2fbdaa51a" (UID: "9759ccec-0bb9-473b-b461-f1e2fbdaa51a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 16:40:53.792715 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:53.792675    2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9759ccec-0bb9-473b-b461-f1e2fbdaa51a-istio-podinfo\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:40:53.792715 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:53.792706    2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9759ccec-0bb9-473b-b461-f1e2fbdaa51a-istio-token\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:40:53.792715 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:53.792715    2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9759ccec-0bb9-473b-b461-f1e2fbdaa51a-workload-socket\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:40:53.792715 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:53.792726    2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9759ccec-0bb9-473b-b461-f1e2fbdaa51a-istiod-ca-cert\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:40:53.792987 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:53.792735    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-stng9\" (UniqueName: \"kubernetes.io/projected/9759ccec-0bb9-473b-b461-f1e2fbdaa51a-kube-api-access-stng9\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:40:53.792987 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:53.792747    2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9759ccec-0bb9-473b-b461-f1e2fbdaa51a-istio-data\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:40:53.792987 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:53.792756    2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9759ccec-0bb9-473b-b461-f1e2fbdaa51a-istio-envoy\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:40:54.223213 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:54.223174    2567 generic.go:356] "Generic (PLEG): container finished" podID="9759ccec-0bb9-473b-b461-f1e2fbdaa51a" containerID="b3e33a842f92ca955a7ee8f41e1d58855a99b08a57733fb2f6a376621c312c84" exitCode=0
Sep 09 16:40:54.223373 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:54.223222    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr"
Sep 09 16:40:54.223373 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:54.223254    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr" event={"ID":"9759ccec-0bb9-473b-b461-f1e2fbdaa51a","Type":"ContainerDied","Data":"b3e33a842f92ca955a7ee8f41e1d58855a99b08a57733fb2f6a376621c312c84"}
Sep 09 16:40:54.223373 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:54.223315    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr" event={"ID":"9759ccec-0bb9-473b-b461-f1e2fbdaa51a","Type":"ContainerDied","Data":"7226d8fc7097d3a50b7d8d70e7bbadd6b8e62e4fffb22184e79cb5f6c37fb13d"}
Sep 09 16:40:54.223373 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:54.223344    2567 scope.go:117] "RemoveContainer" containerID="b3e33a842f92ca955a7ee8f41e1d58855a99b08a57733fb2f6a376621c312c84"
Sep 09 16:40:54.232837 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:54.232809    2567 scope.go:117] "RemoveContainer" containerID="83768d4a8797d376deab3ca8464ea5dfa11e851656769cfde646c91322b2aefa"
Sep 09 16:40:54.241146 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:54.241129    2567 scope.go:117] "RemoveContainer" containerID="b3e33a842f92ca955a7ee8f41e1d58855a99b08a57733fb2f6a376621c312c84"
Sep 09 16:40:54.241407 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:40:54.241389    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3e33a842f92ca955a7ee8f41e1d58855a99b08a57733fb2f6a376621c312c84\": container with ID starting with b3e33a842f92ca955a7ee8f41e1d58855a99b08a57733fb2f6a376621c312c84 not found: ID does not exist" containerID="b3e33a842f92ca955a7ee8f41e1d58855a99b08a57733fb2f6a376621c312c84"
Sep 09 16:40:54.241487 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:54.241413    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3e33a842f92ca955a7ee8f41e1d58855a99b08a57733fb2f6a376621c312c84"} err="failed to get container status \"b3e33a842f92ca955a7ee8f41e1d58855a99b08a57733fb2f6a376621c312c84\": rpc error: code = NotFound desc = could not find container \"b3e33a842f92ca955a7ee8f41e1d58855a99b08a57733fb2f6a376621c312c84\": container with ID starting with b3e33a842f92ca955a7ee8f41e1d58855a99b08a57733fb2f6a376621c312c84 not found: ID does not exist"
Sep 09 16:40:54.241487 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:54.241429    2567 scope.go:117] "RemoveContainer" containerID="83768d4a8797d376deab3ca8464ea5dfa11e851656769cfde646c91322b2aefa"
Sep 09 16:40:54.241715 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:40:54.241693    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83768d4a8797d376deab3ca8464ea5dfa11e851656769cfde646c91322b2aefa\": container with ID starting with 83768d4a8797d376deab3ca8464ea5dfa11e851656769cfde646c91322b2aefa not found: ID does not exist" containerID="83768d4a8797d376deab3ca8464ea5dfa11e851656769cfde646c91322b2aefa"
Sep 09 16:40:54.241845 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:54.241723    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83768d4a8797d376deab3ca8464ea5dfa11e851656769cfde646c91322b2aefa"} err="failed to get container status \"83768d4a8797d376deab3ca8464ea5dfa11e851656769cfde646c91322b2aefa\": rpc error: code = NotFound desc = could not find container \"83768d4a8797d376deab3ca8464ea5dfa11e851656769cfde646c91322b2aefa\": container with ID starting with 83768d4a8797d376deab3ca8464ea5dfa11e851656769cfde646c91322b2aefa not found: ID does not exist"
Sep 09 16:40:54.244830 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:54.244801    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr"]
Sep 09 16:40:54.247786 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:54.247765    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-3c6036-openshift-default-65b574bdb6-5xvzr"]
Sep 09 16:40:55.036296 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:40:55.036259    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9759ccec-0bb9-473b-b461-f1e2fbdaa51a" path="/var/lib/kubelet/pods/9759ccec-0bb9-473b-b461-f1e2fbdaa51a/volumes"
Sep 09 16:41:02.113141 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:02.113098    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:41:17.199473 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:17.199421    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9520a4-cb5556ffd-ndqxc"]
Sep 09 16:41:17.199965 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:17.199715    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9520a4-cb5556ffd-ndqxc" podUID="5e9ca9ce-194e-4c2b-a02a-7b514ecf6385" containerName="maas-api" containerID="cri-o://db24911dbe2115b81d0c6770e95b3d9c2568978fb8c8455d3f654075515b9a6a" gracePeriod=30
Sep 09 16:41:17.338607 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:17.338569    2567 generic.go:356] "Generic (PLEG): container finished" podID="5e9ca9ce-194e-4c2b-a02a-7b514ecf6385" containerID="db24911dbe2115b81d0c6770e95b3d9c2568978fb8c8455d3f654075515b9a6a" exitCode=0
Sep 09 16:41:17.338797 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:17.338648    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9520a4-cb5556ffd-ndqxc" event={"ID":"5e9ca9ce-194e-4c2b-a02a-7b514ecf6385","Type":"ContainerDied","Data":"db24911dbe2115b81d0c6770e95b3d9c2568978fb8c8455d3f654075515b9a6a"}
Sep 09 16:41:17.443120 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:17.443093    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9520a4-cb5556ffd-ndqxc"
Sep 09 16:41:17.529252 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:17.529152    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9llbm\" (UniqueName: \"kubernetes.io/projected/5e9ca9ce-194e-4c2b-a02a-7b514ecf6385-kube-api-access-9llbm\") pod \"5e9ca9ce-194e-4c2b-a02a-7b514ecf6385\" (UID: \"5e9ca9ce-194e-4c2b-a02a-7b514ecf6385\") "
Sep 09 16:41:17.529252 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:17.529201    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e9ca9ce-194e-4c2b-a02a-7b514ecf6385-metrics-certs\") pod \"5e9ca9ce-194e-4c2b-a02a-7b514ecf6385\" (UID: \"5e9ca9ce-194e-4c2b-a02a-7b514ecf6385\") "
Sep 09 16:41:17.529519 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:17.529257    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e9ca9ce-194e-4c2b-a02a-7b514ecf6385-openshift-service-ca\") pod \"5e9ca9ce-194e-4c2b-a02a-7b514ecf6385\" (UID: \"5e9ca9ce-194e-4c2b-a02a-7b514ecf6385\") "
Sep 09 16:41:17.529519 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:17.529277    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5e9ca9ce-194e-4c2b-a02a-7b514ecf6385-maas-api-tls\") pod \"5e9ca9ce-194e-4c2b-a02a-7b514ecf6385\" (UID: \"5e9ca9ce-194e-4c2b-a02a-7b514ecf6385\") "
Sep 09 16:41:17.529716 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:17.529612    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e9ca9ce-194e-4c2b-a02a-7b514ecf6385-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5e9ca9ce-194e-4c2b-a02a-7b514ecf6385" (UID: "5e9ca9ce-194e-4c2b-a02a-7b514ecf6385"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 16:41:17.531658 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:17.531623    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e9ca9ce-194e-4c2b-a02a-7b514ecf6385-kube-api-access-9llbm" (OuterVolumeSpecName: "kube-api-access-9llbm") pod "5e9ca9ce-194e-4c2b-a02a-7b514ecf6385" (UID: "5e9ca9ce-194e-4c2b-a02a-7b514ecf6385"). InnerVolumeSpecName "kube-api-access-9llbm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 16:41:17.531764 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:17.531672    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e9ca9ce-194e-4c2b-a02a-7b514ecf6385-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "5e9ca9ce-194e-4c2b-a02a-7b514ecf6385" (UID: "5e9ca9ce-194e-4c2b-a02a-7b514ecf6385"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 16:41:17.531764 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:17.531690    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e9ca9ce-194e-4c2b-a02a-7b514ecf6385-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5e9ca9ce-194e-4c2b-a02a-7b514ecf6385" (UID: "5e9ca9ce-194e-4c2b-a02a-7b514ecf6385"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 16:41:17.630100 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:17.630059    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e9ca9ce-194e-4c2b-a02a-7b514ecf6385-openshift-service-ca\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:41:17.630274 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:17.630109    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5e9ca9ce-194e-4c2b-a02a-7b514ecf6385-maas-api-tls\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:41:17.630274 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:17.630123    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9llbm\" (UniqueName: \"kubernetes.io/projected/5e9ca9ce-194e-4c2b-a02a-7b514ecf6385-kube-api-access-9llbm\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:41:17.630274 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:17.630135    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e9ca9ce-194e-4c2b-a02a-7b514ecf6385-metrics-certs\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:41:18.350630 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:18.350597    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9520a4-cb5556ffd-ndqxc"
Sep 09 16:41:18.350630 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:18.350616    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9520a4-cb5556ffd-ndqxc" event={"ID":"5e9ca9ce-194e-4c2b-a02a-7b514ecf6385","Type":"ContainerDied","Data":"39acbd085c49da144ca113a0897cffead52ab65fa57da801318672a25e73cfbb"}
Sep 09 16:41:18.351155 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:18.350693    2567 scope.go:117] "RemoveContainer" containerID="db24911dbe2115b81d0c6770e95b3d9c2568978fb8c8455d3f654075515b9a6a"
Sep 09 16:41:18.373906 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:18.373762    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9520a4-cb5556ffd-ndqxc"]
Sep 09 16:41:18.375579 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:18.375556    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9520a4-cb5556ffd-ndqxc"]
Sep 09 16:41:19.035308 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:19.035274    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e9ca9ce-194e-4c2b-a02a-7b514ecf6385" path="/var/lib/kubelet/pods/5e9ca9ce-194e-4c2b-a02a-7b514ecf6385/volumes"
Sep 09 16:41:19.705480 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:19.705430    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:41:24.122435 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:24.122387    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl"]
Sep 09 16:41:24.122971 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:24.122670    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl" podUID="993e8512-0a13-4565-8fd6-53d9f4910d95" containerName="istio-proxy" containerID="cri-o://87307eb8f599b4d36bfe7390cf78211a77ed899f0c89e046572a39954a37acb3" gracePeriod=30
Sep 09 16:41:24.646409 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:24.646372    2567 patch_prober.go:28] interesting pod/e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 09 16:41:24.646628 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:24.646440    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl" podUID="993e8512-0a13-4565-8fd6-53d9f4910d95" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 16:41:28.511900 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:28.511858    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:41:29.380085 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:29.380057    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl"
Sep 09 16:41:29.406792 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:29.406717    2567 generic.go:356] "Generic (PLEG): container finished" podID="993e8512-0a13-4565-8fd6-53d9f4910d95" containerID="87307eb8f599b4d36bfe7390cf78211a77ed899f0c89e046572a39954a37acb3" exitCode=0
Sep 09 16:41:29.406792 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:29.406779    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl" event={"ID":"993e8512-0a13-4565-8fd6-53d9f4910d95","Type":"ContainerDied","Data":"87307eb8f599b4d36bfe7390cf78211a77ed899f0c89e046572a39954a37acb3"}
Sep 09 16:41:29.406979 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:29.406844    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl" event={"ID":"993e8512-0a13-4565-8fd6-53d9f4910d95","Type":"ContainerDied","Data":"6c9131879348bec23ca3d0e4d3282c5a22e76b99705f2bccecaaa557fdf26737"}
Sep 09 16:41:29.406979 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:29.406778    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl"
Sep 09 16:41:29.406979 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:29.406870    2567 scope.go:117] "RemoveContainer" containerID="87307eb8f599b4d36bfe7390cf78211a77ed899f0c89e046572a39954a37acb3"
Sep 09 16:41:29.415814 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:29.415786    2567 scope.go:117] "RemoveContainer" containerID="0bbb8347070570f0310967a60f3d7c7cab59f2c4b33c3d7291097889e04def6c"
Sep 09 16:41:29.425632 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:29.425610    2567 scope.go:117] "RemoveContainer" containerID="87307eb8f599b4d36bfe7390cf78211a77ed899f0c89e046572a39954a37acb3"
Sep 09 16:41:29.425910 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:41:29.425893    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"87307eb8f599b4d36bfe7390cf78211a77ed899f0c89e046572a39954a37acb3\": container with ID starting with 87307eb8f599b4d36bfe7390cf78211a77ed899f0c89e046572a39954a37acb3 not found: ID does not exist" containerID="87307eb8f599b4d36bfe7390cf78211a77ed899f0c89e046572a39954a37acb3"
Sep 09 16:41:29.425972 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:29.425934    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87307eb8f599b4d36bfe7390cf78211a77ed899f0c89e046572a39954a37acb3"} err="failed to get container status \"87307eb8f599b4d36bfe7390cf78211a77ed899f0c89e046572a39954a37acb3\": rpc error: code = NotFound desc = could not find container \"87307eb8f599b4d36bfe7390cf78211a77ed899f0c89e046572a39954a37acb3\": container with ID starting with 87307eb8f599b4d36bfe7390cf78211a77ed899f0c89e046572a39954a37acb3 not found: ID does not exist"
Sep 09 16:41:29.425972 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:29.425951    2567 scope.go:117] "RemoveContainer" containerID="0bbb8347070570f0310967a60f3d7c7cab59f2c4b33c3d7291097889e04def6c"
Sep 09 16:41:29.426233 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:41:29.426214    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0bbb8347070570f0310967a60f3d7c7cab59f2c4b33c3d7291097889e04def6c\": container with ID starting with 0bbb8347070570f0310967a60f3d7c7cab59f2c4b33c3d7291097889e04def6c not found: ID does not exist" containerID="0bbb8347070570f0310967a60f3d7c7cab59f2c4b33c3d7291097889e04def6c"
Sep 09 16:41:29.426275 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:29.426241    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0bbb8347070570f0310967a60f3d7c7cab59f2c4b33c3d7291097889e04def6c"} err="failed to get container status \"0bbb8347070570f0310967a60f3d7c7cab59f2c4b33c3d7291097889e04def6c\": rpc error: code = NotFound desc = could not find container \"0bbb8347070570f0310967a60f3d7c7cab59f2c4b33c3d7291097889e04def6c\": container with ID starting with 0bbb8347070570f0310967a60f3d7c7cab59f2c4b33c3d7291097889e04def6c not found: ID does not exist"
Sep 09 16:41:29.445614 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:29.445581    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/993e8512-0a13-4565-8fd6-53d9f4910d95-workload-certs\") pod \"993e8512-0a13-4565-8fd6-53d9f4910d95\" (UID: \"993e8512-0a13-4565-8fd6-53d9f4910d95\") "
Sep 09 16:41:29.445726 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:29.445623    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/993e8512-0a13-4565-8fd6-53d9f4910d95-istio-podinfo\") pod \"993e8512-0a13-4565-8fd6-53d9f4910d95\" (UID: \"993e8512-0a13-4565-8fd6-53d9f4910d95\") "
Sep 09 16:41:29.445726 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:29.445640    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/993e8512-0a13-4565-8fd6-53d9f4910d95-istio-envoy\") pod \"993e8512-0a13-4565-8fd6-53d9f4910d95\" (UID: \"993e8512-0a13-4565-8fd6-53d9f4910d95\") "
Sep 09 16:41:29.445726 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:29.445660    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/993e8512-0a13-4565-8fd6-53d9f4910d95-istiod-ca-cert\") pod \"993e8512-0a13-4565-8fd6-53d9f4910d95\" (UID: \"993e8512-0a13-4565-8fd6-53d9f4910d95\") "
Sep 09 16:41:29.445726 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:29.445696    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9rk29\" (UniqueName: \"kubernetes.io/projected/993e8512-0a13-4565-8fd6-53d9f4910d95-kube-api-access-9rk29\") pod \"993e8512-0a13-4565-8fd6-53d9f4910d95\" (UID: \"993e8512-0a13-4565-8fd6-53d9f4910d95\") "
Sep 09 16:41:29.445726 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:29.445711    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/993e8512-0a13-4565-8fd6-53d9f4910d95-istio-token\") pod \"993e8512-0a13-4565-8fd6-53d9f4910d95\" (UID: \"993e8512-0a13-4565-8fd6-53d9f4910d95\") "
Sep 09 16:41:29.445962 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:29.445759    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/993e8512-0a13-4565-8fd6-53d9f4910d95-istio-data\") pod \"993e8512-0a13-4565-8fd6-53d9f4910d95\" (UID: \"993e8512-0a13-4565-8fd6-53d9f4910d95\") "
Sep 09 16:41:29.445962 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:29.445792    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/993e8512-0a13-4565-8fd6-53d9f4910d95-credential-socket\") pod \"993e8512-0a13-4565-8fd6-53d9f4910d95\" (UID: \"993e8512-0a13-4565-8fd6-53d9f4910d95\") "
Sep 09 16:41:29.445962 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:29.445830    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/993e8512-0a13-4565-8fd6-53d9f4910d95-workload-socket\") pod \"993e8512-0a13-4565-8fd6-53d9f4910d95\" (UID: \"993e8512-0a13-4565-8fd6-53d9f4910d95\") "
Sep 09 16:41:29.445962 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:29.445888    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/993e8512-0a13-4565-8fd6-53d9f4910d95-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "993e8512-0a13-4565-8fd6-53d9f4910d95" (UID: "993e8512-0a13-4565-8fd6-53d9f4910d95"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:41:29.446209 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:29.446188    2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/993e8512-0a13-4565-8fd6-53d9f4910d95-workload-certs\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:41:29.446373 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:29.446245    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/993e8512-0a13-4565-8fd6-53d9f4910d95-istio-data" (OuterVolumeSpecName: "istio-data") pod "993e8512-0a13-4565-8fd6-53d9f4910d95" (UID: "993e8512-0a13-4565-8fd6-53d9f4910d95"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:41:29.446373 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:29.446346    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/993e8512-0a13-4565-8fd6-53d9f4910d95-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "993e8512-0a13-4565-8fd6-53d9f4910d95" (UID: "993e8512-0a13-4565-8fd6-53d9f4910d95"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 16:41:29.446373 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:29.446300    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/993e8512-0a13-4565-8fd6-53d9f4910d95-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "993e8512-0a13-4565-8fd6-53d9f4910d95" (UID: "993e8512-0a13-4565-8fd6-53d9f4910d95"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:41:29.446736 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:29.446662    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/993e8512-0a13-4565-8fd6-53d9f4910d95-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "993e8512-0a13-4565-8fd6-53d9f4910d95" (UID: "993e8512-0a13-4565-8fd6-53d9f4910d95"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:41:29.448050 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:29.448015    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/993e8512-0a13-4565-8fd6-53d9f4910d95-kube-api-access-9rk29" (OuterVolumeSpecName: "kube-api-access-9rk29") pod "993e8512-0a13-4565-8fd6-53d9f4910d95" (UID: "993e8512-0a13-4565-8fd6-53d9f4910d95"). InnerVolumeSpecName "kube-api-access-9rk29". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 16:41:29.448161 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:29.448094    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/993e8512-0a13-4565-8fd6-53d9f4910d95-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "993e8512-0a13-4565-8fd6-53d9f4910d95" (UID: "993e8512-0a13-4565-8fd6-53d9f4910d95"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 16:41:29.448525 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:29.448498    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/993e8512-0a13-4565-8fd6-53d9f4910d95-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "993e8512-0a13-4565-8fd6-53d9f4910d95" (UID: "993e8512-0a13-4565-8fd6-53d9f4910d95"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 16:41:29.448525 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:29.448506    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/993e8512-0a13-4565-8fd6-53d9f4910d95-istio-token" (OuterVolumeSpecName: "istio-token") pod "993e8512-0a13-4565-8fd6-53d9f4910d95" (UID: "993e8512-0a13-4565-8fd6-53d9f4910d95"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 16:41:29.547446 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:29.547410    2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/993e8512-0a13-4565-8fd6-53d9f4910d95-istiod-ca-cert\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:41:29.547446 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:29.547441    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9rk29\" (UniqueName: \"kubernetes.io/projected/993e8512-0a13-4565-8fd6-53d9f4910d95-kube-api-access-9rk29\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:41:29.547446 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:29.547475    2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/993e8512-0a13-4565-8fd6-53d9f4910d95-istio-token\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:41:29.547933 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:29.547485    2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/993e8512-0a13-4565-8fd6-53d9f4910d95-istio-data\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:41:29.547933 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:29.547494    2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/993e8512-0a13-4565-8fd6-53d9f4910d95-credential-socket\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:41:29.547933 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:29.547502    2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/993e8512-0a13-4565-8fd6-53d9f4910d95-workload-socket\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:41:29.547933 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:29.547510    2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/993e8512-0a13-4565-8fd6-53d9f4910d95-istio-podinfo\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:41:29.547933 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:29.547517    2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/993e8512-0a13-4565-8fd6-53d9f4910d95-istio-envoy\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:41:29.728353 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:29.728318    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl"]
Sep 09 16:41:29.731787 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:29.731759    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-9520a4-openshift-default-9d68d8496-d2rxl"]
Sep 09 16:41:31.036011 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:41:31.035972    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="993e8512-0a13-4565-8fd6-53d9f4910d95" path="/var/lib/kubelet/pods/993e8512-0a13-4565-8fd6-53d9f4910d95/volumes"
Sep 09 16:42:52.707308 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:42:52.707270    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:43:23.604114 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:43:23.604082    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:43:31.510508 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:43:31.510473    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:43:39.106287 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:43:39.106246    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:43:45.201036 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:43:45.201009    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-xwlzs_16ecfb4d-7944-465e-a38e-d8c5bda6b1ac/console-operator/2.log"
Sep 09 16:43:45.214562 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:43:45.214540    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-xwlzs_16ecfb4d-7944-465e-a38e-d8c5bda6b1ac/console-operator/2.log"
Sep 09 16:43:46.013998 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:43:46.013960    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:44:00.299564 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:44:00.299527    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:44:10.408476 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:44:10.408432    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:45:39.414285 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:45:39.414250    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:45:46.316665 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:45:46.316625    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:45:56.223879 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:45:56.223841    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:46:04.123656 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:04.123625    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:46:13.217995 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:13.217914    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:46:28.624966 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:28.624932    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:46:37.715761 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:37.715725    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:46:44.837497 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:44.837467    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7b7b9ff5c-b5qvg"]
Sep 09 16:46:44.838049 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:44.837860    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9759ccec-0bb9-473b-b461-f1e2fbdaa51a" containerName="istio-proxy"
Sep 09 16:46:44.838049 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:44.837877    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9759ccec-0bb9-473b-b461-f1e2fbdaa51a" containerName="istio-proxy"
Sep 09 16:46:44.838049 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:44.837896    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e5b5a13-128c-4713-bd42-fb9de49b5838" containerName="istio-proxy"
Sep 09 16:46:44.838049 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:44.837906    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e5b5a13-128c-4713-bd42-fb9de49b5838" containerName="istio-proxy"
Sep 09 16:46:44.838049 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:44.837917    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="993e8512-0a13-4565-8fd6-53d9f4910d95" containerName="istio-proxy"
Sep 09 16:46:44.838049 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:44.837925    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="993e8512-0a13-4565-8fd6-53d9f4910d95" containerName="istio-proxy"
Sep 09 16:46:44.838049 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:44.837938    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="447f2a14-e79a-4ffa-b257-0eb0d7cf82c7" containerName="maas-api"
Sep 09 16:46:44.838049 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:44.837946    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="447f2a14-e79a-4ffa-b257-0eb0d7cf82c7" containerName="maas-api"
Sep 09 16:46:44.838049 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:44.837965    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e5b5a13-128c-4713-bd42-fb9de49b5838" containerName="istio-proxy"
Sep 09 16:46:44.838049 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:44.837972    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e5b5a13-128c-4713-bd42-fb9de49b5838" containerName="istio-proxy"
Sep 09 16:46:44.838049 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:44.837984    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e9ca9ce-194e-4c2b-a02a-7b514ecf6385" containerName="maas-api"
Sep 09 16:46:44.838049 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:44.837992    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e9ca9ce-194e-4c2b-a02a-7b514ecf6385" containerName="maas-api"
Sep 09 16:46:44.838049 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:44.838001    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="993e8512-0a13-4565-8fd6-53d9f4910d95" containerName="istio-proxy"
Sep 09 16:46:44.838049 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:44.838009    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="993e8512-0a13-4565-8fd6-53d9f4910d95" containerName="istio-proxy"
Sep 09 16:46:44.838049 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:44.838030    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99b1c6ad-1a67-40f5-a39d-3c23c6693e42" containerName="maas-api"
Sep 09 16:46:44.838049 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:44.838037    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="99b1c6ad-1a67-40f5-a39d-3c23c6693e42" containerName="maas-api"
Sep 09 16:46:44.838746 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:44.838127    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="993e8512-0a13-4565-8fd6-53d9f4910d95" containerName="istio-proxy"
Sep 09 16:46:44.838746 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:44.838153    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e5b5a13-128c-4713-bd42-fb9de49b5838" containerName="istio-proxy"
Sep 09 16:46:44.838746 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:44.838163    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="9759ccec-0bb9-473b-b461-f1e2fbdaa51a" containerName="istio-proxy"
Sep 09 16:46:44.838746 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:44.838171    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="447f2a14-e79a-4ffa-b257-0eb0d7cf82c7" containerName="maas-api"
Sep 09 16:46:44.838746 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:44.838182    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="993e8512-0a13-4565-8fd6-53d9f4910d95" containerName="istio-proxy"
Sep 09 16:46:44.838746 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:44.838190    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="99b1c6ad-1a67-40f5-a39d-3c23c6693e42" containerName="maas-api"
Sep 09 16:46:44.838746 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:44.838197    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e9ca9ce-194e-4c2b-a02a-7b514ecf6385" containerName="maas-api"
Sep 09 16:46:44.838746 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:44.838205    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="9759ccec-0bb9-473b-b461-f1e2fbdaa51a" containerName="istio-proxy"
Sep 09 16:46:44.841353 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:44.841334    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7b7b9ff5c-b5qvg"
Sep 09 16:46:44.844679 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:44.844635    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-mslnv\""
Sep 09 16:46:44.844679 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:44.844668    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 09 16:46:44.844861 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:44.844709    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 09 16:46:44.844861 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:44.844731    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 09 16:46:44.844861 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:44.844670    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 09 16:46:44.853786 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:44.853763    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7b7b9ff5c-b5qvg"]
Sep 09 16:46:44.967532 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:44.967503    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a87baa02-d162-4540-bbd0-3c5274bc94cd-maas-api-tls\") pod \"maas-api-7b7b9ff5c-b5qvg\" (UID: \"a87baa02-d162-4540-bbd0-3c5274bc94cd\") " pod="odh-ai-gateway-infra/maas-api-7b7b9ff5c-b5qvg"
Sep 09 16:46:44.967671 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:44.967620    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a87baa02-d162-4540-bbd0-3c5274bc94cd-openshift-service-ca\") pod \"maas-api-7b7b9ff5c-b5qvg\" (UID: \"a87baa02-d162-4540-bbd0-3c5274bc94cd\") " pod="odh-ai-gateway-infra/maas-api-7b7b9ff5c-b5qvg"
Sep 09 16:46:44.967671 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:44.967644    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a87baa02-d162-4540-bbd0-3c5274bc94cd-metrics-certs\") pod \"maas-api-7b7b9ff5c-b5qvg\" (UID: \"a87baa02-d162-4540-bbd0-3c5274bc94cd\") " pod="odh-ai-gateway-infra/maas-api-7b7b9ff5c-b5qvg"
Sep 09 16:46:44.967752 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:44.967671    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdnvq\" (UniqueName: \"kubernetes.io/projected/a87baa02-d162-4540-bbd0-3c5274bc94cd-kube-api-access-bdnvq\") pod \"maas-api-7b7b9ff5c-b5qvg\" (UID: \"a87baa02-d162-4540-bbd0-3c5274bc94cd\") " pod="odh-ai-gateway-infra/maas-api-7b7b9ff5c-b5qvg"
Sep 09 16:46:45.068692 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:45.068663    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a87baa02-d162-4540-bbd0-3c5274bc94cd-openshift-service-ca\") pod \"maas-api-7b7b9ff5c-b5qvg\" (UID: \"a87baa02-d162-4540-bbd0-3c5274bc94cd\") " pod="odh-ai-gateway-infra/maas-api-7b7b9ff5c-b5qvg"
Sep 09 16:46:45.068692 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:45.068695    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a87baa02-d162-4540-bbd0-3c5274bc94cd-metrics-certs\") pod \"maas-api-7b7b9ff5c-b5qvg\" (UID: \"a87baa02-d162-4540-bbd0-3c5274bc94cd\") " pod="odh-ai-gateway-infra/maas-api-7b7b9ff5c-b5qvg"
Sep 09 16:46:45.068914 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:45.068717    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdnvq\" (UniqueName: \"kubernetes.io/projected/a87baa02-d162-4540-bbd0-3c5274bc94cd-kube-api-access-bdnvq\") pod \"maas-api-7b7b9ff5c-b5qvg\" (UID: \"a87baa02-d162-4540-bbd0-3c5274bc94cd\") " pod="odh-ai-gateway-infra/maas-api-7b7b9ff5c-b5qvg"
Sep 09 16:46:45.068914 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:45.068738    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a87baa02-d162-4540-bbd0-3c5274bc94cd-maas-api-tls\") pod \"maas-api-7b7b9ff5c-b5qvg\" (UID: \"a87baa02-d162-4540-bbd0-3c5274bc94cd\") " pod="odh-ai-gateway-infra/maas-api-7b7b9ff5c-b5qvg"
Sep 09 16:46:45.071066 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:45.071042    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 09 16:46:45.071185 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:45.071044    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 09 16:46:45.071185 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:45.071102    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 09 16:46:45.076030 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:45.076014    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 09 16:46:45.080297 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:45.080277    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a87baa02-d162-4540-bbd0-3c5274bc94cd-openshift-service-ca\") pod \"maas-api-7b7b9ff5c-b5qvg\" (UID: \"a87baa02-d162-4540-bbd0-3c5274bc94cd\") " pod="odh-ai-gateway-infra/maas-api-7b7b9ff5c-b5qvg"
Sep 09 16:46:45.081868 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:45.081849    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a87baa02-d162-4540-bbd0-3c5274bc94cd-maas-api-tls\") pod \"maas-api-7b7b9ff5c-b5qvg\" (UID: \"a87baa02-d162-4540-bbd0-3c5274bc94cd\") " pod="odh-ai-gateway-infra/maas-api-7b7b9ff5c-b5qvg"
Sep 09 16:46:45.081964 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:45.081945    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a87baa02-d162-4540-bbd0-3c5274bc94cd-metrics-certs\") pod \"maas-api-7b7b9ff5c-b5qvg\" (UID: \"a87baa02-d162-4540-bbd0-3c5274bc94cd\") " pod="odh-ai-gateway-infra/maas-api-7b7b9ff5c-b5qvg"
Sep 09 16:46:45.086976 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:45.086952    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdnvq\" (UniqueName: \"kubernetes.io/projected/a87baa02-d162-4540-bbd0-3c5274bc94cd-kube-api-access-bdnvq\") pod \"maas-api-7b7b9ff5c-b5qvg\" (UID: \"a87baa02-d162-4540-bbd0-3c5274bc94cd\") " pod="odh-ai-gateway-infra/maas-api-7b7b9ff5c-b5qvg"
Sep 09 16:46:45.156728 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:45.156679    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-mslnv\""
Sep 09 16:46:45.163343 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:45.163327    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7b7b9ff5c-b5qvg"
Sep 09 16:46:45.417640 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:45.417568    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:46:45.438739 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:45.438712    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7b7b9ff5c-b5qvg"]
Sep 09 16:46:45.441738 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:46:45.441710    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda87baa02_d162_4540_bbd0_3c5274bc94cd.slice/crio-27b9441578005961d0589408694e79668f69d9c9a16b24c4040b584f3cd10ac0 WatchSource:0}: Error finding container 27b9441578005961d0589408694e79668f69d9c9a16b24c4040b584f3cd10ac0: Status 404 returned error can't find the container with id 27b9441578005961d0589408694e79668f69d9c9a16b24c4040b584f3cd10ac0
Sep 09 16:46:45.442828 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:45.442807    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 16:46:45.839421 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:45.839387    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7b7b9ff5c-b5qvg" event={"ID":"a87baa02-d162-4540-bbd0-3c5274bc94cd","Type":"ContainerStarted","Data":"27b9441578005961d0589408694e79668f69d9c9a16b24c4040b584f3cd10ac0"}
Sep 09 16:46:47.849669 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:47.849634    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7b7b9ff5c-b5qvg" event={"ID":"a87baa02-d162-4540-bbd0-3c5274bc94cd","Type":"ContainerStarted","Data":"565af0912bc1588dd88848e6150997c65c0daed2825e3aa17d10a46c5882e317"}
Sep 09 16:46:47.850051 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:47.849721    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7b7b9ff5c-b5qvg"
Sep 09 16:46:47.866734 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:47.866689    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7b7b9ff5c-b5qvg" podStartSLOduration=2.467097684 podStartE2EDuration="3.866674945s" totalImagesPullingTime="1.399577261s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 16:46:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 16:46:47.865777763 +0000 UTC m=+2283.283627763" watchObservedRunningTime="2026-09-09 16:46:47.866674945 +0000 UTC m=+2283.284524951"
Sep 09 16:46:52.514019 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:52.513979    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:46:53.860096 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:53.860068    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7b7b9ff5c-b5qvg"
Sep 09 16:46:54.927311 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:54.927274    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6cfd8fccdf-kkcrr"]
Sep 09 16:46:54.927709 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:54.927689    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9759ccec-0bb9-473b-b461-f1e2fbdaa51a" containerName="istio-proxy"
Sep 09 16:46:54.927709 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:54.927703    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9759ccec-0bb9-473b-b461-f1e2fbdaa51a" containerName="istio-proxy"
Sep 09 16:46:54.927784 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:54.927755    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e5b5a13-128c-4713-bd42-fb9de49b5838" containerName="istio-proxy"
Sep 09 16:46:54.931013 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:54.930996    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6cfd8fccdf-kkcrr"
Sep 09 16:46:54.942262 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:54.942030    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6cfd8fccdf-kkcrr"]
Sep 09 16:46:55.046228 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:55.046195    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96kfn\" (UniqueName: \"kubernetes.io/projected/a60b79c8-060e-4a9b-b557-a6d2a6149d36-kube-api-access-96kfn\") pod \"maas-api-6cfd8fccdf-kkcrr\" (UID: \"a60b79c8-060e-4a9b-b557-a6d2a6149d36\") " pod="odh-ai-gateway-infra/maas-api-6cfd8fccdf-kkcrr"
Sep 09 16:46:55.046357 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:55.046232    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a60b79c8-060e-4a9b-b557-a6d2a6149d36-maas-api-tls\") pod \"maas-api-6cfd8fccdf-kkcrr\" (UID: \"a60b79c8-060e-4a9b-b557-a6d2a6149d36\") " pod="odh-ai-gateway-infra/maas-api-6cfd8fccdf-kkcrr"
Sep 09 16:46:55.046357 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:55.046268    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a60b79c8-060e-4a9b-b557-a6d2a6149d36-metrics-certs\") pod \"maas-api-6cfd8fccdf-kkcrr\" (UID: \"a60b79c8-060e-4a9b-b557-a6d2a6149d36\") " pod="odh-ai-gateway-infra/maas-api-6cfd8fccdf-kkcrr"
Sep 09 16:46:55.046357 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:55.046331    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a60b79c8-060e-4a9b-b557-a6d2a6149d36-openshift-service-ca\") pod \"maas-api-6cfd8fccdf-kkcrr\" (UID: \"a60b79c8-060e-4a9b-b557-a6d2a6149d36\") " pod="odh-ai-gateway-infra/maas-api-6cfd8fccdf-kkcrr"
Sep 09 16:46:55.147142 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:55.147114    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a60b79c8-060e-4a9b-b557-a6d2a6149d36-openshift-service-ca\") pod \"maas-api-6cfd8fccdf-kkcrr\" (UID: \"a60b79c8-060e-4a9b-b557-a6d2a6149d36\") " pod="odh-ai-gateway-infra/maas-api-6cfd8fccdf-kkcrr"
Sep 09 16:46:55.147264 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:55.147155    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96kfn\" (UniqueName: \"kubernetes.io/projected/a60b79c8-060e-4a9b-b557-a6d2a6149d36-kube-api-access-96kfn\") pod \"maas-api-6cfd8fccdf-kkcrr\" (UID: \"a60b79c8-060e-4a9b-b557-a6d2a6149d36\") " pod="odh-ai-gateway-infra/maas-api-6cfd8fccdf-kkcrr"
Sep 09 16:46:55.147264 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:55.147178    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a60b79c8-060e-4a9b-b557-a6d2a6149d36-maas-api-tls\") pod \"maas-api-6cfd8fccdf-kkcrr\" (UID: \"a60b79c8-060e-4a9b-b557-a6d2a6149d36\") " pod="odh-ai-gateway-infra/maas-api-6cfd8fccdf-kkcrr"
Sep 09 16:46:55.147390 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:55.147367    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a60b79c8-060e-4a9b-b557-a6d2a6149d36-metrics-certs\") pod \"maas-api-6cfd8fccdf-kkcrr\" (UID: \"a60b79c8-060e-4a9b-b557-a6d2a6149d36\") " pod="odh-ai-gateway-infra/maas-api-6cfd8fccdf-kkcrr"
Sep 09 16:46:55.147791 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:55.147772    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a60b79c8-060e-4a9b-b557-a6d2a6149d36-openshift-service-ca\") pod \"maas-api-6cfd8fccdf-kkcrr\" (UID: \"a60b79c8-060e-4a9b-b557-a6d2a6149d36\") " pod="odh-ai-gateway-infra/maas-api-6cfd8fccdf-kkcrr"
Sep 09 16:46:55.149881 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:55.149858    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a60b79c8-060e-4a9b-b557-a6d2a6149d36-maas-api-tls\") pod \"maas-api-6cfd8fccdf-kkcrr\" (UID: \"a60b79c8-060e-4a9b-b557-a6d2a6149d36\") " pod="odh-ai-gateway-infra/maas-api-6cfd8fccdf-kkcrr"
Sep 09 16:46:55.149963 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:55.149858    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a60b79c8-060e-4a9b-b557-a6d2a6149d36-metrics-certs\") pod \"maas-api-6cfd8fccdf-kkcrr\" (UID: \"a60b79c8-060e-4a9b-b557-a6d2a6149d36\") " pod="odh-ai-gateway-infra/maas-api-6cfd8fccdf-kkcrr"
Sep 09 16:46:55.154173 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:55.154152    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-96kfn\" (UniqueName: \"kubernetes.io/projected/a60b79c8-060e-4a9b-b557-a6d2a6149d36-kube-api-access-96kfn\") pod \"maas-api-6cfd8fccdf-kkcrr\" (UID: \"a60b79c8-060e-4a9b-b557-a6d2a6149d36\") " pod="odh-ai-gateway-infra/maas-api-6cfd8fccdf-kkcrr"
Sep 09 16:46:55.240863 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:55.240807    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6cfd8fccdf-kkcrr"
Sep 09 16:46:55.512772 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:55.512743    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6cfd8fccdf-kkcrr"]
Sep 09 16:46:55.515033 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:46:55.515001    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda60b79c8_060e_4a9b_b557_a6d2a6149d36.slice/crio-44566a5ab4811522b66f9e3659bf2aa7bcf0906636391d4efb8b10c98fa3ce61 WatchSource:0}: Error finding container 44566a5ab4811522b66f9e3659bf2aa7bcf0906636391d4efb8b10c98fa3ce61: Status 404 returned error can't find the container with id 44566a5ab4811522b66f9e3659bf2aa7bcf0906636391d4efb8b10c98fa3ce61
Sep 09 16:46:55.888910 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:55.888884    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6cfd8fccdf-kkcrr" event={"ID":"a60b79c8-060e-4a9b-b557-a6d2a6149d36","Type":"ContainerStarted","Data":"44566a5ab4811522b66f9e3659bf2aa7bcf0906636391d4efb8b10c98fa3ce61"}
Sep 09 16:46:56.894603 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:56.894567    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6cfd8fccdf-kkcrr" event={"ID":"a60b79c8-060e-4a9b-b557-a6d2a6149d36","Type":"ContainerStarted","Data":"181d1d101a622b65d803d24b642bfde03b95804a73dfa6f5ab603f7695e5a6ec"}
Sep 09 16:46:56.895047 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:56.894628    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6cfd8fccdf-kkcrr"
Sep 09 16:46:56.915418 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:56.915369    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6cfd8fccdf-kkcrr" podStartSLOduration=2.500085943 podStartE2EDuration="2.915354545s" totalImagesPullingTime="415.268602ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 16:46:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 16:46:56.913086201 +0000 UTC m=+2292.330936199" watchObservedRunningTime="2026-09-09 16:46:56.915354545 +0000 UTC m=+2292.333204581"
Sep 09 16:46:59.316723 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:46:59.316691    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:47:02.904825 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:02.904794    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6cfd8fccdf-kkcrr"
Sep 09 16:47:02.955425 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:02.955355    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-7b7b9ff5c-b5qvg"]
Sep 09 16:47:02.955668 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:02.955617    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-7b7b9ff5c-b5qvg" podUID="a87baa02-d162-4540-bbd0-3c5274bc94cd" containerName="maas-api" containerID="cri-o://565af0912bc1588dd88848e6150997c65c0daed2825e3aa17d10a46c5882e317" gracePeriod=30
Sep 09 16:47:03.855711 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:03.855674    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-7b7b9ff5c-b5qvg" podUID="a87baa02-d162-4540-bbd0-3c5274bc94cd" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.60:8443/health\": dial tcp 10.134.0.60:8443: connect: connection refused"
Sep 09 16:47:04.197062 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:04.197038    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7b7b9ff5c-b5qvg"
Sep 09 16:47:04.325317 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:04.325282    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a87baa02-d162-4540-bbd0-3c5274bc94cd-openshift-service-ca\") pod \"a87baa02-d162-4540-bbd0-3c5274bc94cd\" (UID: \"a87baa02-d162-4540-bbd0-3c5274bc94cd\") "
Sep 09 16:47:04.325494 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:04.325335    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a87baa02-d162-4540-bbd0-3c5274bc94cd-metrics-certs\") pod \"a87baa02-d162-4540-bbd0-3c5274bc94cd\" (UID: \"a87baa02-d162-4540-bbd0-3c5274bc94cd\") "
Sep 09 16:47:04.325494 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:04.325446    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bdnvq\" (UniqueName: \"kubernetes.io/projected/a87baa02-d162-4540-bbd0-3c5274bc94cd-kube-api-access-bdnvq\") pod \"a87baa02-d162-4540-bbd0-3c5274bc94cd\" (UID: \"a87baa02-d162-4540-bbd0-3c5274bc94cd\") "
Sep 09 16:47:04.325591 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:04.325511    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a87baa02-d162-4540-bbd0-3c5274bc94cd-maas-api-tls\") pod \"a87baa02-d162-4540-bbd0-3c5274bc94cd\" (UID: \"a87baa02-d162-4540-bbd0-3c5274bc94cd\") "
Sep 09 16:47:04.325733 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:04.325697    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a87baa02-d162-4540-bbd0-3c5274bc94cd-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a87baa02-d162-4540-bbd0-3c5274bc94cd" (UID: "a87baa02-d162-4540-bbd0-3c5274bc94cd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 16:47:04.327719 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:04.327694    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a87baa02-d162-4540-bbd0-3c5274bc94cd-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a87baa02-d162-4540-bbd0-3c5274bc94cd" (UID: "a87baa02-d162-4540-bbd0-3c5274bc94cd"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 16:47:04.327841 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:04.327720    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a87baa02-d162-4540-bbd0-3c5274bc94cd-kube-api-access-bdnvq" (OuterVolumeSpecName: "kube-api-access-bdnvq") pod "a87baa02-d162-4540-bbd0-3c5274bc94cd" (UID: "a87baa02-d162-4540-bbd0-3c5274bc94cd"). InnerVolumeSpecName "kube-api-access-bdnvq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 16:47:04.328036 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:04.328021    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a87baa02-d162-4540-bbd0-3c5274bc94cd-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a87baa02-d162-4540-bbd0-3c5274bc94cd" (UID: "a87baa02-d162-4540-bbd0-3c5274bc94cd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 16:47:04.426951 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:04.426897    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bdnvq\" (UniqueName: \"kubernetes.io/projected/a87baa02-d162-4540-bbd0-3c5274bc94cd-kube-api-access-bdnvq\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:47:04.426951 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:04.426917    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a87baa02-d162-4540-bbd0-3c5274bc94cd-maas-api-tls\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:47:04.426951 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:04.426927    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a87baa02-d162-4540-bbd0-3c5274bc94cd-openshift-service-ca\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:47:04.426951 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:04.426935    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a87baa02-d162-4540-bbd0-3c5274bc94cd-metrics-certs\") on node \"ip-10-0-128-81.ec2.internal\" DevicePath \"\""
Sep 09 16:47:04.930356 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:04.930320    2567 generic.go:356] "Generic (PLEG): container finished" podID="a87baa02-d162-4540-bbd0-3c5274bc94cd" containerID="565af0912bc1588dd88848e6150997c65c0daed2825e3aa17d10a46c5882e317" exitCode=0
Sep 09 16:47:04.930592 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:04.930373    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7b7b9ff5c-b5qvg"
Sep 09 16:47:04.930592 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:04.930410    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7b7b9ff5c-b5qvg" event={"ID":"a87baa02-d162-4540-bbd0-3c5274bc94cd","Type":"ContainerDied","Data":"565af0912bc1588dd88848e6150997c65c0daed2825e3aa17d10a46c5882e317"}
Sep 09 16:47:04.930592 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:04.930532    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7b7b9ff5c-b5qvg" event={"ID":"a87baa02-d162-4540-bbd0-3c5274bc94cd","Type":"ContainerDied","Data":"27b9441578005961d0589408694e79668f69d9c9a16b24c4040b584f3cd10ac0"}
Sep 09 16:47:04.930592 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:04.930561    2567 scope.go:117] "RemoveContainer" containerID="565af0912bc1588dd88848e6150997c65c0daed2825e3aa17d10a46c5882e317"
Sep 09 16:47:04.939726 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:04.939712    2567 scope.go:117] "RemoveContainer" containerID="565af0912bc1588dd88848e6150997c65c0daed2825e3aa17d10a46c5882e317"
Sep 09 16:47:04.939941 ip-10-0-128-81 kubenswrapper[2567]: E0909 16:47:04.939923    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"565af0912bc1588dd88848e6150997c65c0daed2825e3aa17d10a46c5882e317\": container with ID starting with 565af0912bc1588dd88848e6150997c65c0daed2825e3aa17d10a46c5882e317 not found: ID does not exist" containerID="565af0912bc1588dd88848e6150997c65c0daed2825e3aa17d10a46c5882e317"
Sep 09 16:47:04.939991 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:04.939949    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"565af0912bc1588dd88848e6150997c65c0daed2825e3aa17d10a46c5882e317"} err="failed to get container status \"565af0912bc1588dd88848e6150997c65c0daed2825e3aa17d10a46c5882e317\": rpc error: code = NotFound desc = could not find container \"565af0912bc1588dd88848e6150997c65c0daed2825e3aa17d10a46c5882e317\": container with ID starting with 565af0912bc1588dd88848e6150997c65c0daed2825e3aa17d10a46c5882e317 not found: ID does not exist"
Sep 09 16:47:04.951013 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:04.950993    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-7b7b9ff5c-b5qvg"]
Sep 09 16:47:04.954250 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:04.954231    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-7b7b9ff5c-b5qvg"]
Sep 09 16:47:05.035258 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:05.035234    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a87baa02-d162-4540-bbd0-3c5274bc94cd" path="/var/lib/kubelet/pods/a87baa02-d162-4540-bbd0-3c5274bc94cd/volumes"
Sep 09 16:47:05.411179 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:05.411152    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:47:12.213285 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:12.213253    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-n7dqd"]
Sep 09 16:47:21.049594 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:21.049566    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-6cmtl_390a6dc0-1968-427b-b3b8-cc15b1201765/server/0.log"
Sep 09 16:47:21.177658 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:21.177633    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-9b4w6_d385db74-c01b-40d2-89f0-f71cca32b2c4/manager/0.log"
Sep 09 16:47:21.594766 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:21.594742    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-67c7bcf47f-h6rbf_c91ca7cb-43cc-439a-90e4-15c989659880/copy-manifests/0.log"
Sep 09 16:47:21.601601 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:21.601582    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-67c7bcf47f-h6rbf_c91ca7cb-43cc-439a-90e4-15c989659880/manager/0.log"
Sep 09 16:47:21.962868 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:21.962798    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-6cfd8fccdf-kkcrr_a60b79c8-060e-4a9b-b557-a6d2a6149d36/maas-api/0.log"
Sep 09 16:47:22.212030 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:22.212003    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29816190-kr87p_9f8262a7-078f-450a-b5a8-5ee0480da094/cleanup/0.log"
Sep 09 16:47:23.957292 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:23.957263    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-n7dqd_e7f48af3-8fb3-4bb1-91f4-09478ea358f2/limitador/0.log"
Sep 09 16:47:24.459950 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:24.459918    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w_123f0ac2-1909-428d-a00e-3c9bff13a4f2/istio-proxy/0.log"
Sep 09 16:47:24.589067 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:24.589042    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-t9mz5_aeddd9d7-76d4-41a8-baad-a13ce3180a50/discovery/0.log"
Sep 09 16:47:25.337339 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:25.337302    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7d7756ccd6-mfpk8_5ae67a1e-f25a-4218-a9ce-d4614dd65d43/router/0.log"
Sep 09 16:47:25.817826 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:25.817800    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-vhgp4_a5394245-74ab-4f80-9e3a-9b16493e13e5/main/0.log"
Sep 09 16:47:26.309984 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:26.309904    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-979n2_8e7d0d70-2fbc-4f2b-8e12-45103aaf9446/main/0.log"
Sep 09 16:47:37.665287 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:37.665204    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-wk4dc_28f47047-e391-44cb-836e-df79db9e4057/global-pull-secret-syncer/0.log"
Sep 09 16:47:37.762129 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:37.762102    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-m7xl5_83598a1c-9f92-44aa-afa9-42556f6c0127/konnectivity-agent/0.log"
Sep 09 16:47:37.812730 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:37.812704    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-81.ec2.internal_fd9da504dd49d348bc16b790735150bc/haproxy/0.log"
Sep 09 16:47:41.918854 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:41.918771    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-n7dqd_e7f48af3-8fb3-4bb1-91f4-09478ea358f2/limitador/0.log"
Sep 09 16:47:44.012321 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:44.012285    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-c47648b58-qxzjc_8be9e6b1-71f9-4055-914c-66b813d981ec/metrics-server/0.log"
Sep 09 16:47:44.242815 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:44.242781    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xwz7w_32e8c12d-3457-493b-b124-519623fbe249/node-exporter/0.log"
Sep 09 16:47:44.267976 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:44.267905    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xwz7w_32e8c12d-3457-493b-b124-519623fbe249/kube-rbac-proxy/0.log"
Sep 09 16:47:44.308208 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:44.308188    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xwz7w_32e8c12d-3457-493b-b124-519623fbe249/init-textfile/0.log"
Sep 09 16:47:44.370938 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:44.370919    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-6b5vr_18af883a-dc08-484a-984a-6db038a684ed/kube-rbac-proxy-main/0.log"
Sep 09 16:47:44.430370 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:44.430341    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-6b5vr_18af883a-dc08-484a-984a-6db038a684ed/kube-rbac-proxy-self/0.log"
Sep 09 16:47:44.454566 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:44.454548    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-6b5vr_18af883a-dc08-484a-984a-6db038a684ed/openshift-state-metrics/0.log"
Sep 09 16:47:44.497021 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:44.496996    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fefe2c92-8751-43f2-b904-d80d04c9340a/prometheus/0.log"
Sep 09 16:47:44.519089 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:44.519022    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fefe2c92-8751-43f2-b904-d80d04c9340a/config-reloader/0.log"
Sep 09 16:47:44.540210 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:44.540192    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fefe2c92-8751-43f2-b904-d80d04c9340a/thanos-sidecar/0.log"
Sep 09 16:47:44.561193 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:44.561178    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fefe2c92-8751-43f2-b904-d80d04c9340a/kube-rbac-proxy-web/0.log"
Sep 09 16:47:44.582383 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:44.582362    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fefe2c92-8751-43f2-b904-d80d04c9340a/kube-rbac-proxy/0.log"
Sep 09 16:47:44.615549 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:44.615529    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fefe2c92-8751-43f2-b904-d80d04c9340a/kube-rbac-proxy-thanos/0.log"
Sep 09 16:47:44.638066 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:44.638040    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fefe2c92-8751-43f2-b904-d80d04c9340a/init-config-reloader/0.log"
Sep 09 16:47:44.671160 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:44.671141    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-8tttv_10c36f54-5efc-4f1b-aba7-7e4e3116aea7/prometheus-operator/0.log"
Sep 09 16:47:44.699382 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:44.699360    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-8tttv_10c36f54-5efc-4f1b-aba7-7e4e3116aea7/kube-rbac-proxy/0.log"
Sep 09 16:47:44.746947 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:44.746924    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-78f599db79-m5hq5_d6c84114-6c4c-4a95-95c3-057ec1ddf0e3/prometheus-operator-admission-webhook/0.log"
Sep 09 16:47:47.272550 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:47.272506    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-xwlzs_16ecfb4d-7944-465e-a38e-d8c5bda6b1ac/console-operator/2.log"
Sep 09 16:47:47.279792 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:47.279763    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-xwlzs_16ecfb4d-7944-465e-a38e-d8c5bda6b1ac/console-operator/3.log"
Sep 09 16:47:48.215646 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:48.215611    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-jjpd4/perf-node-gather-daemonset-vpvnr"]
Sep 09 16:47:48.216044 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:48.216019    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a87baa02-d162-4540-bbd0-3c5274bc94cd" containerName="maas-api"
Sep 09 16:47:48.216044 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:48.216043    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a87baa02-d162-4540-bbd0-3c5274bc94cd" containerName="maas-api"
Sep 09 16:47:48.216248 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:48.216163    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="a87baa02-d162-4540-bbd0-3c5274bc94cd" containerName="maas-api"
Sep 09 16:47:48.220324 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:48.220308    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jjpd4/perf-node-gather-daemonset-vpvnr"
Sep 09 16:47:48.222466 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:48.222436    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jjpd4\"/\"kube-root-ca.crt\""
Sep 09 16:47:48.223229 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:48.223210    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jjpd4\"/\"openshift-service-ca.crt\""
Sep 09 16:47:48.223327 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:48.223213    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-jjpd4\"/\"default-dockercfg-glm2f\""
Sep 09 16:47:48.228868 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:48.228840    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jjpd4/perf-node-gather-daemonset-vpvnr"]
Sep 09 16:47:48.262673 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:48.262643    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d5e6420f-710d-4945-a5fc-43481a4691ac-lib-modules\") pod \"perf-node-gather-daemonset-vpvnr\" (UID: \"d5e6420f-710d-4945-a5fc-43481a4691ac\") " pod="openshift-must-gather-jjpd4/perf-node-gather-daemonset-vpvnr"
Sep 09 16:47:48.262816 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:48.262683    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d5e6420f-710d-4945-a5fc-43481a4691ac-sys\") pod \"perf-node-gather-daemonset-vpvnr\" (UID: \"d5e6420f-710d-4945-a5fc-43481a4691ac\") " pod="openshift-must-gather-jjpd4/perf-node-gather-daemonset-vpvnr"
Sep 09 16:47:48.262816 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:48.262754    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d5e6420f-710d-4945-a5fc-43481a4691ac-podres\") pod \"perf-node-gather-daemonset-vpvnr\" (UID: \"d5e6420f-710d-4945-a5fc-43481a4691ac\") " pod="openshift-must-gather-jjpd4/perf-node-gather-daemonset-vpvnr"
Sep 09 16:47:48.262816 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:48.262808    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5k69\" (UniqueName: \"kubernetes.io/projected/d5e6420f-710d-4945-a5fc-43481a4691ac-kube-api-access-s5k69\") pod \"perf-node-gather-daemonset-vpvnr\" (UID: \"d5e6420f-710d-4945-a5fc-43481a4691ac\") " pod="openshift-must-gather-jjpd4/perf-node-gather-daemonset-vpvnr"
Sep 09 16:47:48.262948 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:48.262849    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d5e6420f-710d-4945-a5fc-43481a4691ac-proc\") pod \"perf-node-gather-daemonset-vpvnr\" (UID: \"d5e6420f-710d-4945-a5fc-43481a4691ac\") " pod="openshift-must-gather-jjpd4/perf-node-gather-daemonset-vpvnr"
Sep 09 16:47:48.363449 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:48.363402    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d5e6420f-710d-4945-a5fc-43481a4691ac-lib-modules\") pod \"perf-node-gather-daemonset-vpvnr\" (UID: \"d5e6420f-710d-4945-a5fc-43481a4691ac\") " pod="openshift-must-gather-jjpd4/perf-node-gather-daemonset-vpvnr"
Sep 09 16:47:48.363857 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:48.363483    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d5e6420f-710d-4945-a5fc-43481a4691ac-sys\") pod \"perf-node-gather-daemonset-vpvnr\" (UID: \"d5e6420f-710d-4945-a5fc-43481a4691ac\") " pod="openshift-must-gather-jjpd4/perf-node-gather-daemonset-vpvnr"
Sep 09 16:47:48.363857 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:48.363530    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d5e6420f-710d-4945-a5fc-43481a4691ac-podres\") pod \"perf-node-gather-daemonset-vpvnr\" (UID: \"d5e6420f-710d-4945-a5fc-43481a4691ac\") " pod="openshift-must-gather-jjpd4/perf-node-gather-daemonset-vpvnr"
Sep 09 16:47:48.363857 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:48.363562    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5k69\" (UniqueName: \"kubernetes.io/projected/d5e6420f-710d-4945-a5fc-43481a4691ac-kube-api-access-s5k69\") pod \"perf-node-gather-daemonset-vpvnr\" (UID: \"d5e6420f-710d-4945-a5fc-43481a4691ac\") " pod="openshift-must-gather-jjpd4/perf-node-gather-daemonset-vpvnr"
Sep 09 16:47:48.363857 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:48.363579    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d5e6420f-710d-4945-a5fc-43481a4691ac-lib-modules\") pod \"perf-node-gather-daemonset-vpvnr\" (UID: \"d5e6420f-710d-4945-a5fc-43481a4691ac\") " pod="openshift-must-gather-jjpd4/perf-node-gather-daemonset-vpvnr"
Sep 09 16:47:48.363857 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:48.363591    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d5e6420f-710d-4945-a5fc-43481a4691ac-sys\") pod \"perf-node-gather-daemonset-vpvnr\" (UID: \"d5e6420f-710d-4945-a5fc-43481a4691ac\") " pod="openshift-must-gather-jjpd4/perf-node-gather-daemonset-vpvnr"
Sep 09 16:47:48.363857 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:48.363587    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d5e6420f-710d-4945-a5fc-43481a4691ac-proc\") pod \"perf-node-gather-daemonset-vpvnr\" (UID: \"d5e6420f-710d-4945-a5fc-43481a4691ac\") " pod="openshift-must-gather-jjpd4/perf-node-gather-daemonset-vpvnr"
Sep 09 16:47:48.363857 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:48.363621    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d5e6420f-710d-4945-a5fc-43481a4691ac-proc\") pod \"perf-node-gather-daemonset-vpvnr\" (UID: \"d5e6420f-710d-4945-a5fc-43481a4691ac\") " pod="openshift-must-gather-jjpd4/perf-node-gather-daemonset-vpvnr"
Sep 09 16:47:48.363857 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:48.363707    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d5e6420f-710d-4945-a5fc-43481a4691ac-podres\") pod \"perf-node-gather-daemonset-vpvnr\" (UID: \"d5e6420f-710d-4945-a5fc-43481a4691ac\") " pod="openshift-must-gather-jjpd4/perf-node-gather-daemonset-vpvnr"
Sep 09 16:47:48.371724 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:48.371701    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5k69\" (UniqueName: \"kubernetes.io/projected/d5e6420f-710d-4945-a5fc-43481a4691ac-kube-api-access-s5k69\") pod \"perf-node-gather-daemonset-vpvnr\" (UID: \"d5e6420f-710d-4945-a5fc-43481a4691ac\") " pod="openshift-must-gather-jjpd4/perf-node-gather-daemonset-vpvnr"
Sep 09 16:47:48.532638 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:48.532553    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jjpd4/perf-node-gather-daemonset-vpvnr"
Sep 09 16:47:48.802786 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:48.802748    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jjpd4/perf-node-gather-daemonset-vpvnr"]
Sep 09 16:47:48.805428 ip-10-0-128-81 kubenswrapper[2567]: W0909 16:47:48.805400    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podd5e6420f_710d_4945_a5fc_43481a4691ac.slice/crio-9e640b55af85910908868e5ae038b267f5402d08c7437f5b9223112557d8632f WatchSource:0}: Error finding container 9e640b55af85910908868e5ae038b267f5402d08c7437f5b9223112557d8632f: Status 404 returned error can't find the container with id 9e640b55af85910908868e5ae038b267f5402d08c7437f5b9223112557d8632f
Sep 09 16:47:49.134350 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:49.134310    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jjpd4/perf-node-gather-daemonset-vpvnr" event={"ID":"d5e6420f-710d-4945-a5fc-43481a4691ac","Type":"ContainerStarted","Data":"301047c4f9e4cee4baa7465f0ce5267fcc4ad529fc436cfb45136267f5cdda98"}
Sep 09 16:47:49.134553 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:49.134371    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jjpd4/perf-node-gather-daemonset-vpvnr" event={"ID":"d5e6420f-710d-4945-a5fc-43481a4691ac","Type":"ContainerStarted","Data":"9e640b55af85910908868e5ae038b267f5402d08c7437f5b9223112557d8632f"}
Sep 09 16:47:49.134553 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:49.134402    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-jjpd4/perf-node-gather-daemonset-vpvnr"
Sep 09 16:47:49.150765 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:49.150715    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-jjpd4/perf-node-gather-daemonset-vpvnr" podStartSLOduration=1.150702101 podStartE2EDuration="1.150702101s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 16:47:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 16:47:49.149064349 +0000 UTC m=+2344.566914360" watchObservedRunningTime="2026-09-09 16:47:49.150702101 +0000 UTC m=+2344.568552100"
Sep 09 16:47:49.895652 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:49.895620    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-frbps_8f8cbff3-9dfd-4406-9c22-260bd366d946/dns/0.log"
Sep 09 16:47:49.913733 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:49.913708    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-frbps_8f8cbff3-9dfd-4406-9c22-260bd366d946/kube-rbac-proxy/0.log"
Sep 09 16:47:49.963439 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:49.963407    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-cnvzz_deb44589-97aa-464f-95b0-61362f120948/dns-node-resolver/0.log"
Sep 09 16:47:50.535614 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:50.535587    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-c7kvc_f996fce2-faf6-474a-997d-8a9419e788d8/node-ca/0.log"
Sep 09 16:47:51.531551 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:51.531525    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w_123f0ac2-1909-428d-a00e-3c9bff13a4f2/istio-proxy/0.log"
Sep 09 16:47:51.943182 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:51.943154    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-t9mz5_aeddd9d7-76d4-41a8-baad-a13ce3180a50/discovery/0.log"
Sep 09 16:47:52.134196 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:52.134162    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7d7756ccd6-mfpk8_5ae67a1e-f25a-4218-a9ce-d4614dd65d43/router/0.log"
Sep 09 16:47:53.052726 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:53.052696    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tbdg5_65ebbfa4-b996-49d0-ae3e-2a1b8c3d7677/serve-healthcheck-canary/0.log"
Sep 09 16:47:53.624759 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:53.624729    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lg6v8_cade2714-b3c7-4de0-841b-b2f3bc582e6c/kube-rbac-proxy/0.log"
Sep 09 16:47:53.646674 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:53.646645    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lg6v8_cade2714-b3c7-4de0-841b-b2f3bc582e6c/exporter/0.log"
Sep 09 16:47:53.668058 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:53.668034    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lg6v8_cade2714-b3c7-4de0-841b-b2f3bc582e6c/extractor/0.log"
Sep 09 16:47:55.149587 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:55.149557    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-jjpd4/perf-node-gather-daemonset-vpvnr"
Sep 09 16:47:56.283706 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:56.283677    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-6cmtl_390a6dc0-1968-427b-b3b8-cc15b1201765/server/0.log"
Sep 09 16:47:56.405031 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:56.404996    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-9b4w6_d385db74-c01b-40d2-89f0-f71cca32b2c4/manager/0.log"
Sep 09 16:47:56.579066 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:56.579037    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-67c7bcf47f-h6rbf_c91ca7cb-43cc-439a-90e4-15c989659880/manager/0.log"
Sep 09 16:47:56.602851 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:56.602825    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-67c7bcf47f-h6rbf_c91ca7cb-43cc-439a-90e4-15c989659880/copy-manifests/0.log"
Sep 09 16:47:58.031978 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:58.031933    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7bc87557cd-6bpm8_83e7f88c-6264-429c-a506-f98dce47d2f7/manager/0.log"
Sep 09 16:47:58.441410 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:58.441378    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w_123f0ac2-1909-428d-a00e-3c9bff13a4f2/istio-proxy/0.log"
Sep 09 16:47:58.838332 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:58.838296    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-t9mz5_aeddd9d7-76d4-41a8-baad-a13ce3180a50/discovery/0.log"
Sep 09 16:47:58.974020 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:47:58.973990    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7d7756ccd6-mfpk8_5ae67a1e-f25a-4218-a9ce-d4614dd65d43/router/0.log"
Sep 09 16:48:00.472414 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:48:00.472375    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-grj5w_123f0ac2-1909-428d-a00e-3c9bff13a4f2/istio-proxy/0.log"
Sep 09 16:48:00.879582 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:48:00.879555    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-t9mz5_aeddd9d7-76d4-41a8-baad-a13ce3180a50/discovery/0.log"
Sep 09 16:48:01.020818 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:48:01.020784    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7d7756ccd6-mfpk8_5ae67a1e-f25a-4218-a9ce-d4614dd65d43/router/0.log"
Sep 09 16:48:04.841789 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:48:04.841761    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-l82fl_15f41781-3a63-422e-9355-e37c98cc7c07/kube-multus-additional-cni-plugins/0.log"
Sep 09 16:48:04.863192 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:48:04.863154    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-l82fl_15f41781-3a63-422e-9355-e37c98cc7c07/egress-router-binary-copy/0.log"
Sep 09 16:48:04.886902 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:48:04.886873    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-l82fl_15f41781-3a63-422e-9355-e37c98cc7c07/cni-plugins/0.log"
Sep 09 16:48:04.906266 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:48:04.906243    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-l82fl_15f41781-3a63-422e-9355-e37c98cc7c07/bond-cni-plugin/0.log"
Sep 09 16:48:04.924294 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:48:04.924276    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-l82fl_15f41781-3a63-422e-9355-e37c98cc7c07/routeoverride-cni/0.log"
Sep 09 16:48:04.945584 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:48:04.945559    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-l82fl_15f41781-3a63-422e-9355-e37c98cc7c07/whereabouts-cni-bincopy/0.log"
Sep 09 16:48:04.964462 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:48:04.964439    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-l82fl_15f41781-3a63-422e-9355-e37c98cc7c07/whereabouts-cni/0.log"
Sep 09 16:48:05.206054 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:48:05.205969    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-q2pgn_36d2c95e-5f46-413b-a6f5-4f0c4bd57b05/kube-multus/0.log"
Sep 09 16:48:05.388599 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:48:05.388572    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-t29nl_73b8ff58-aa37-4fab-9632-9b0f5d409902/network-metrics-daemon/0.log"
Sep 09 16:48:05.407220 ip-10-0-128-81 kubenswrapper[2567]: I0909 16:48:05.407196    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-t29nl_73b8ff58-aa37-4fab-9632-9b0f5d409902/kube-rbac-proxy/0.log"